MANAGEMENT INFORMATION SYSTEMS

DATA MINING AND BUSINESS INTELLIGENCE

Week 8 – Artificial Neural Network

PROF. DR. GÖKHAN SILAHTAROĞLU
Lecturer:. NADA MISK
 • Artificial neural networks are machine learning
algorithms that are inspired by the working principles of
the human brain..
• Why ANN
Artificial Neural ➢ Knowledge acquisition under noise and uncertainty.
Network ➢ Ability to derive meaning from complicated or
imprecise data.
➢ Extract patterns and detect trends that are too
complex to be noticed by either humans or other
computer techniques.
➢ Adaptive learning and fault tolerance.
➢ Flexible Knowledge representation.
➢ Efficient knowledge processing.
➢ Real-Time processing.
 BIOLOGICAL NEURON STRUCTURE

• The brain consists of 1011

neurons as in the picture on
the right.

https://www.youtube.com/watch?v=b2ctEsGEpe0

https://www.youtube.com/watch?v=qPix_X-9t7E
 • An Artificial Neural Network is a network of
interconnected artificial neurons, where each
neuron represents a computing unit.
• These neurons interact with each other and are
connected in various ways.
• Each node receives input and performs some
operations on it before transmitting.

ARTIFICIAL NEURAL • Its most distinctive features are; neurons

connected to each other, the determination of
NETWORKS weights between connections, and the activation
function.

Input Hidden Output

Mathematical
Model of A
Human Neuron
(ANN Model)
Black Box

• From an engineering point of view,

ANN can be seen as a "black box".
• The black box receives the
information from the outside and
gives the outputs it produces to the
outside. What is inside is unknown.
• In other words, ANN does not have
the ability to explain how it
generates results. Although this
situation undermines the trust in the
network, successful applications
constantly increase the interest in
artificial neural networks.
Elements of ANN Model

• Input
It is the information coming from the outside world or another cell to the artificial neural networks.
• Weghits
It represents the numerical value of the connections between cells. It shows the value of the information received in a cell
and its effect on the cell.
• Summation Function
It provides the calculation of the net input of that cell by multiplying the inputs to the cell with the weights and summing
them.
• Activation Function
It processes the net input to the cell and determines the output that the cell will produce in response to this input.
• Output
They are the output values determined by the activation functions. The produced output can be sent either to the outside
world, to another cell, or as an input to itself.
Perceptron, The Keystone

• Perceptron is an algorithm
that mimics the biological
neuron.
• Its components: Inputs,
weights of inputs, sum of
weights function, activation
function, activation threshold
and output.
THE PERCEPTRON : FORWARD PROPAGATION
THE PERCEPTRON : FORWARD PROPAGATION
ACTIVATION FUNCTION

y=Activation(∑(w*x+b))
The activation function is used here to control the y value, that is, to decide whether a neuron will be active or not.
Step Function:Produces a binary classification output (0 or 1) based on a threshold value.
Sigmoid Function : It is one of the most widely used activation functions, it produces output in the range of [0,1].
Tanh Function : It is a nonlinear function that produces output in the range [-1,1].
ReLU Function : Rectified Linear Unit (RELU) is a nonlinear function. The ReLU function takes the value 0 for
negative inputs, while x takes the value x for positive inputs.
Softplus Function :
ELU Function :
PReLU
ACTIVATION FUNCTION

 Activation functions are functions that decide what the node's output should be, given the inputs in
the node.
 We often refer to a layer's outputs as "activations", as it is the activation function that decides the
actual output.
 This function returns 0 if the linear combination is less than 0. Returns 1 if linear combination is
positive or equal to zero (Step function).
 Activation Functions
▪ In order for the y neuron to become active, the y-input value must reach a certain
threshold value accompanied by a function.
▪ So, y=f(y-input). The most widely used activation function is the S-shaped logistic
1
sigmoid function. f ( x) =
1 + e −x

e x − e−x
Another activation function is for all x values.; f ( x) = x
e + e−x

1 if x 
f ( x) =  
Hard transition activation function;
0 if x 
Sigmoid Activation Function

1
f ( x) =
1 + e −x
Sigmoid
1,2
1
0,8
0,6
0,4
0,2
0
-5 -4 -3 -2 -1 0 1 2 3 4 5
X
 Hard Transition Activation Function(step Function)

Sert Geçişli
1,2
1

1 if x    0,8
f ( x) =  
0 if x   
0,6
0,4
0,2
0
-5 -4 -3 -2 -1 0 1 2 3 4 5
x
Hyperbolic Tangent Activation Function

Hiperbolik Tanjant
1,5

1,0

e x − e−x
f ( x) = x
0,5

e + e−x 0,0
-5 -4 -4 -3 -3 -2 -2 -1 -1 -0 0 0, 1 1, 2 2, 3 3, 4 4, 5
-0,5

-1,0

-1,5
x
Activation
Functions
WEGHITS (W)

 When input data arrives at a neuron, it is multiplied by a weight value assigned to that particular
input.
 Weight usage:
 These weights start out as random values, and as the neural network learns more about what kind
of input data leads to a student being admitted to the university, the network adjusts the weights
for any categorization errors caused by the previous weights. This process is to train the neural
network.
 Let's relate the weight to m (slope) in the original linear equation.
 Artificial Neural Network
X1 Z1

Considering the Y neuron w1 wy1

here, this neuron receives Y

signals from the X1, X2 and X2

w2
X3 neurons. The weights
connecting X1, X2 and X3 wy2
w3
neurons to Y neurons are X3 Gizli
given as w1, w2 and w3, Katman
Z2
respectively.
Girdi
Katmanı Çıktı
Katmanı

𝑦 = 𝑥1 𝑤1 + 𝑥2 𝑤2 +𝑥3 𝑤3
ANN Models • Feed Forward
Cells are in the form of regular layers from the entrance to
According to the exit. The information coming to the network passes
through the input layer, then through the hidden layers and
their the output layer, respectively, and then goes out to the
outside world.
Structures • Feedback
The output of a cell is not only given as input to the layer that
follows it. It can be given as an input to any cell in the
previous layer or in its own layer.
Feed Forward & Feedback ANNs
 FEED FORWARD ANN
The w weights used in feed forward artificial neural networks are corrected and
renewed w each time.

= w1 + w1
new old
w1

1 2 1
Er = e = ( g − y) 2
2 2
Learning coefficient that takes
a value in the range of 0 - 1

Er
w = −
w
Feed Forward ANN
Hidden Layer Output Layer

wijgizli

j j

S hidden
j , O hidden
j S ioutput , O ioutput
Cross section of hidden and output layers in neural network

Oicik ti =  ( S icik ti )
n gizli

S i
cikti
= w gizli
ij O gizli
j
j =1
Feed Forward
Feed Forward
Example
Feed Forward - Example
Feed Forward - Example

Continue same way:

Feed Forward
 Feed Forward

Same way:
 Feed Forward

Calculating Error

It is then put for differential calculation.

Will be required when powers are cancelled.
 Feed Forward

Previously calculated
 Feed Forward

Is calculated and:

Total Error
Backpropagation Algorithm
- error in hidden layers
The ideas of the algorithm can be summarized as
follows:

1. Calculates the error term for the output units (in

the last layer) using the observed error.
2. Repeat from output layer
- propagate the error term back to the previous layer,
and
- updating the weights between the two layers until
the oldest hidden layer is reached.
Backpropagation
Backpropagation
Backpropagation
ANN working structure - In summary

Identification of samples

Determination of the topology of the network

• Determining the number of inputs and outputs

Determining the learning parameters of the network

• learning coefficient and determination of constants

Assigning initial values of the network

Up to the number of epochs:

• For all samples in the training set
• The sample is shown to the network
• Calculate the error
• Updating the weights according to the error found

The total error of the system is calculated.

ANNs Features

• They have the ability to self-organize and learn.

• They can work with incomplete information.
• They have fault tolerance.
• They can process vague and incomplete
information.
• ANNs do not show sudden deterioration.
• They can only work with numerical information.
• ANNs were designed to solve problems that are
difficult to solve by normal ways.
ANNs Advantages

• It is successful in solving non-linear, multidimensional,

noisy, incomplete problems, and especially in the
absence of a precise mathematical model or algorithm
for solving the problem.
• It gives good results in cases with a large number of
exceptional and anomalous data.
• It has the ability to adapt.
• The information is stored throughout the network.
• It can generate information about previously unseen
samples.
ANNs Disdvantages

• The 'Black Box' cannot explain its result.

• There are no specific rules in determining the
appropriate network structure.
• There is no certain rule in determining the parameter
values of the network.
• There is no general rule in the selection of training
samples.
• The representation of the problem to be learned to
the network is an important problem.
• There is no specific method for when the training of
the network should be completed.