TO Artificial Neural Networks

INTRODUCTION
TO
ARTIFICIAL NEURAL NETWORKS
CONTENTS
INTRODUCTION
BIOLOGICAL NEURON MODEL
ARTIFICIAL NEURON MODEL
ARTIFICIAL NEURAL NETWORK
NEURAL NETWORK ARCHITECTURE
LEARNING
BACKPROPAGATION ALGORITHM
APPLICATIONS
ADVANTAGES
CONCLUSION
INTRODUCTION
Neural is an adjective for neuron, and network denotes a graph like
structure.
Artificial Neural Networks are also referred to as neural nets , artificial
neural systems, parallel distributed processing systems, connectionist
systems.
For a computing systems to be called by these pretty names, it is necessary
for the system to have a labeled directed graph structure where nodes
performs some simple computations.
Directed Graph consists of set of nodes(vertices) and a set of
connections(edges/links/arcs) connecting pair of nodes.
A graph is said to be labeled graph if each connection is associated with a
label to identify some property of the connection
CONTD multiplier
x1{0,1}
x1
w1 (x1 w1)
x1 x2
(x2w2)
o = x1 AND x2 o = x1 AND x2
w2
x2{0,1} x2
Fig 1: AND gate graph Fig 2: AND gate network

This graph cannot be considered a neural The graph structure which connects the
network since the connections between the weights modifiable using a learning
nodes are fixed and appear to play no other algorithm, qualifies the computing
role than carrying the inputs to the node that system to be called an artificial neural
computed their conjunction. networks.
The field of neural network was pioneered by BERNARD WIDROW of Stanford University in
1950s.
BIOLOGICAL NEURON MODEL
Four parts of a typical nerve cell : -

DENDRITES: Accepts the inputs
SOMA : Process the inputs
AXON : Turns the processed inputs into
outputs.
SYNAPSES : The electrochemical
contact between the
neurons.
ARTIFICIAL NEURON MODEL
Inputs to the network are represented by the
x1
mathematical symbol, xn w1
Each of these inputs are multiplied by a w2 f(w1 x1 + + wnxn)

connection weight , wn x2
f
sum = w1 x1 + + wnxn
These products are simply summed, fed wn
through the transfer function, f( ) to generate
xn
a result and then output.
TERMINOLOGY
Biological Terminology Artificial Neural Network Terminology
Neuron Node/Unit/Cell/Neurode
Synapse Connection/Edge/Link
Synaptic Efficiency Connection Strength/Weight
Firing frequency Node output

ARTIFICIAL NEURAL NETWORK
Artificial Neural Network (ANNs) are programs designed to solve
any problem by trying to mimic the structure and the function of our
nervous system.
Neural networks are based on simulated neurons, Which are joined
together in a variety of ways to form networks.
Neural network resembles the human brain in the following two
ways: -
* A neural network acquires knowledge through learning.
*A neural networks knowledge is stored within the
interconnection strengths known as synaptic weight.
CONTD
ARTIFICIAL NEURAL NETWORK MODEL
Hidden layers
connections Desired
output
Neural network
Input Actual
Including output
output layer output
connections Comp
(called weights) are
between neuron
Input layer
Fig 1 : artificial neural network model Figure showing adjust of neural network
NEURAL NETWORK ARCHITECTURES
Hidden node
Input node
Input node output node

Layer0 Layer3
(Input layer) (Output layer)
output node Layer 1 Layer2
Hidden Layer
Fig: fully connected network
fig: layered network
The neural network in which every node is
connected to every other nodes, and
these connections may be either These are networks in which nodes are
excitatory (positive weights), inhibitory partitioned into subsets called layers, with
(negative weights), or irrelevant (almost no connections from layer j to k if j > k.
zero weights).
CONTD
Layer0 Layer3
(Input layer) (Output layer) Layer0 Layer3
Layer 1 Layer2
(Input layer) (Output layer)
Layer 1 Layer2
Hidden Layer
Hidden Layer
Fig : Acyclic network fig : Feedforward network
This is the subclass of the layered
networks in which there is no intra-layer
connections. In other words, a This is a subclass of acyclic
connection may exist between any node networks in which a connection is
in layer i and any node in layer j for i < j, allowed from a node in layer i only
but a connection is not allowed for i=j. to nodes in layer i+1
CONTD
Fig : Modular neural network
Many problems are best solved using neural networks whose

architecture consists of several modules, with sparse
interconnections between them. Modules can be organized in
several different ways as Hierarchial organization, Successive
refinement, Input modularity
LEARNING
Neurons in an animals brain are hard wired. It is equally obvious that
animals, especially higher order animals, learn as they grow.
How does this learning occur?
What are possible mathematical models of learning?
In artificial neural networks, learning refers to the method of
modifying the weights of connections between the nodes of a
specified network.
The learning ability of a neural network is determined by its
architecture and by the algorithmic method chosen for training.
CONTD
SU PE R VISED L E A R NING U N SU PE RVISED L E A RNING
A teacher is available to indicate This is learning by doing.

whether a system is performing
correctly, or to indicate the amount of In this approach no sample
error in system performance. Here a outputs are provided to the
teacher is a set of training data. network against which it can
The training data consist of pairs of measure its predictive
input and desired output values that performance for a given vector of
are traditionally represented in data inputs.
vectors.
One common form of
Supervised learning can also be unsupervised learning is
referred as classification, where we clustering where we try to
have a wide range of classifiers,
(Multilayer perceptron, k nearest categorize data in different
neighbor..etc) clusters by their similarity.
THE BACKPROPAGATION ALGORITHM
The backpropagation algorithm (Rumelhart and McClelland, 1986) is

used in layered feed-forward Artificial Neural Networks.
Back propagation is a multi-layer feed forward, supervised learning
network based on gradient descent learning rule.
we provide the algorithm with examples of the inputs and outputs we
want the network to compute, and then the error (difference
between actual and expected results) is calculated.
The idea of the backpropagation algorithm is to reduce this error,
until the Artificial Neural Network learns the training data.
The activation function of the artificial neurons in ANNs
implementing the backpropagation algorithm is a
weighted sum (the sum of the inputs xi multiplied by their
respective weights wji) Inputs, x
output
The most common output function is the sigmoidal

function:
Since the error is the difference between the actual and

Weights, v weights, w
the desired output, the error depends on the weights, and
we need to adjust the weights in order to minimize the
error. We can define the error function for the output of Fig: Basic Block of
Back propagation neural network
each neuron:
CONTD
The backpropagation algorithm now calculates how the error depends on the
output, inputs, and weights.
the adjustment of each weight (wji ) will be the negative of a constant eta ()
multiplied by the dependance of the wji previous weight on the error of the network.
First, we need to calculate how much the error depends on the output
Next, how much the output depends on the activation, which in turn depends on the
weights
And so, the adjustment to each weight will be

CONTD
If we want to adjust vik, the weights (lets call them vik ) of a previous
layer, we need first to calculate how the error depends not on the
weight, but in the input from the previous layer i.e. replacing w by x
as shown in below equation.
Inputs, x
output
where
and
Weights, v weights, w
NEURAL NETWORK
APPLICATIONS
ADVANTAGES
It involves human like thinking.
They handle noisy or missing data.
They can work with large number of variables or parameters.
They provide general solutions with good predictive accuracy.
System has got property of continuous learning.
They deal with the non-linearity in the world in which we live.
CONCLUSION

TO Artificial Neural Networks

Uploaded by

Copyright:

Available Formats

TO Artificial Neural Networks

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

TO Artificial Neural Networks

Uploaded by

Copyright:

Available Formats

INTRODUCTION

Fig 1: AND gate graph Fig 2: AND gate network

Four parts of a typical nerve cell : -

Each of these inputs are multiplied by a w2 f(w1 x1 + + wnxn)

Biological Terminology Artificial Neural Network Terminology

Synaptic Efficiency Connection Strength/Weight

Firing frequency Node output

ARTIFICIAL NEURAL NETWORK MODEL

Input node output node

Fig : Modular neural network

Many problems are best solved using neural networks whose

SU PE R VISED L E A R NING U N SU PE RVISED L E A RNING

A teacher is available to indicate This is learning by doing.

The backpropagation algorithm (Rumelhart and McClelland, 1986) is

The most common output function is the sigmoidal

Since the error is the difference between the actual and

And so, the adjustment to each weight will be

You might also like