Ann Basics PDF

Foundations of Articial Neural Networks
Bernd Ulmann Edinburgh, 04-JUN-2003
04-JUN-2003
What are artical neural networks (ANN)?

A new paradigm of computing. A simplied implementation of biologically inspired neural
networks.
Where ANNs are better than traditional computers

Adapting to new environments by learning. Processing fuzzy (imprecise) data. Working with noisy or erroneous data. Performing classication tasks very quickly.
Foundations of Articial Neural Networks 04-JUN-2003 2
Biological neural networks the basics

A human brain contains about
neurons.

Each neuron has inputs (synapses connected to dendrites) and

an output (axon ending in synapses).
One neuron generates input for
other neurons. neurons.
Each neuron receives data from
Synapses connect neurons to each other. Signals are transmitted by releasing chemicals
(neurotransmitters) from a sender neuron causing an electric potential in the receiving cell, etc.
Foundations of Articial Neural Networks 04-JUN-2003 3
Biological neurons
Dendrite
Excitatory synapses Inhibitory synapses Nucleus Myelin coating Cell body Axon
04-JUN-2003
Simplied model of a neuron

A neuron has a current activation All inputs get numeric data

from other neurons. .

Inputs are multiplied by weights

.
Generate a net input by employing a propagation rule:

Calculated new activation ( is a threshold value)

act
A new output is calculated using an output function:

out

.
04-JUN-2003 5
A simple feed forward ANN

Input layer
First hidden layer
Second hidden layer
Output layer
04-JUN-2003
Basic types of ANNs

Feed forward ANNs
Several layers of neurons are connected unidirectionally. Connections can skip layers.

Feedback ANNs
Direct feedback networks allow a neuron to have a feedback connection to itself self amplication/attenuation.
Indirect feedback networks have only feedback connections between layers a method to control attention of the ANN.
Lateral feedback networks have feedback connections only within a layer winner takes all network.
04-JUN-2003 7
Implementing ANNs
Synchronous activation: All neurons change their states
simultaneously very useful for SIMD parallel computers.
Asynchronous activation:
Fixed order activation: Order of activation is xed. Random order activation: Order is (pseudo)random not every neuron is calculated, some are calculated multiply. Random permutation activation: As before but every neuron is calcuted once and only once. Topological order activation: Ideal for feedforward networks.
04-JUN-2003
Training (teaching) ANNs by

creating new connections/neurons, deleting existing connections/neurons, changing the weights
, or by and/or
out
changing the threshold changing the functions

In most cases, the weights
used to simulate additional or deleted connections/neurons.

act
will be modied this can also be
04-JUN-2003
Basic learning strategies

Supervised learning : The learning algorithm receives the output
generated by the ANN as well as the desired output and performs changes on the weights . Desired result has to be known
in advance, sometimes not possible/realistic.
Reinforcement learning : The ANN receives results in form of

reward/punishment from actions based on its generated output.
Unsupervised learning : The only goal is to build representations

of the input data which can in turn be used for prediction/classication, etc.
04-JUN-2003
10
The backpropagation learning rule

Idea: Change the weights the actual output

according to the difference between with a

somewhat complex mechanism as follows: with for output neurons
and the teaching value
act act
for hidden neurons
04-JUN-2003
11
Input data sets

Training set : Examples used to train the ANN (for example by
changing the weights according to the backpropagation rule).
Validation set : Data set for tuning the ANN (for example by
changing the number of hidden layers, etc.).
Test set : Data set used to assess the performance of the ANN
(obviously never to be used as a training set). Sometimes the difference between the training and the validation set is not quite clear and depends on the applied learning rule.
04-JUN-2003
12
References
[1] Klaus Peter Kratzer, Neuronale Netze, Grundlagen und Anwendungen, Carl Hanser Verlag 1991. [2] Armando Freitas da Rocha, Neural Nets, A Theory for Brains and Machines, Lecture Notes in Articial Intelligence 638, Springer-Verlag 1992. [3] Raul Rojas, Theorie der neuronalen Netze Eine systematische Einfuhrung, 4., korrigierter Nachdruck, Springer-Verlag 1996. [4] Warren S. Sarle, ai-faq/neural-nets/part1, etc.,
ftp://ftp.sas.com/pub/neural/FAQ.html.
[5] Andreas Zell, Simulation Neuronaler Netze, Addision-Wesley publishing company, 1994.
12-1

Ann Basics PDF

Uploaded by

Copyright:

Available Formats

Ann Basics PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ann Basics PDF

Uploaded by

Copyright:

Available Formats

Foundations of Articial Neural Networks

Bernd Ulmann Edinburgh, 04-JUN-2003

Foundations of Articial Neural Networks

What are artical neural networks (ANN)?

Where ANNs are better than traditional computers

Biological neural networks the basics

Each neuron has inputs (synapses connected to dendrites) and

One neuron generates input for

other neurons. neurons.

Each neuron receives data from

Foundations of Articial Neural Networks

Simplied model of a neuron

from other neurons. .

Inputs are multiplied by weights

Generate a net input by employing a propagation rule:

Calculated new activation ( is a threshold value)

A new output is calculated using an output function:

A simple feed forward ANN

First hidden layer

Second hidden layer

Foundations of Articial Neural Networks

Basic types of ANNs

Foundations of Articial Neural Networks

Foundations of Articial Neural Networks

Training (teaching) ANNs by

changing the threshold changing the functions

used to simulate additional or deleted connections/neurons.

will be modied this can also be

Foundations of Articial Neural Networks

Basic learning strategies

in advance, sometimes not possible/realistic.

Reinforcement learning : The ANN receives results in form of

Unsupervised learning : The only goal is to build representations

Foundations of Articial Neural Networks

The backpropagation learning rule

according to the difference between with a

somewhat complex mechanism as follows: with for output neurons

and the teaching value

Foundations of Articial Neural Networks

for hidden neurons

Input data sets

Foundations of Articial Neural Networks

You might also like