Scribd Logo
Upload

Welcome to Scribd!

  • 99 views

    NN Adaline

    Uploaded by

    Khasmad Ariansyah
    The document discusses neural networks and the Adaline algorithm. It provides an overview of the topics to be covered, including Perceptrons, calculating the Adaline algorithm for AND and XOR functions, and deriving the Least Mean Square (LMS) algorithm. The LMS algorithm is then explained in steps. Applications of Adaline are mentioned, including better convergence than Perceptrons and use in noise correction. Limitations are discussed, such as only working with linearly separable data, and how multi-level neural networks can address this.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PPT, PDF, TXT or read online from Scribd

    NN Adaline

    Uploaded by

    Khasmad Ariansyah
    99 views14 pages
    The document discusses neural networks and the Adaline algorithm. It provides an overview of the topics to be covered, including Perceptrons, calculating the Adaline algorithm for AND and XOR functions, and deriving the Least Mean Square (LMS) algorithm. The LMS algorithm is then explained in steps. Applications of Adaline are mentioned, including better convergence than Perceptrons and use in noise correction. Limitations are discussed, such as only working with linearly separable data, and how multi-level neural networks can address this.

    Original Description:

    Adaline

    Original Title

    NN.adaline

    Copyright

    © Attribution Non-Commercial (BY-NC)

    Available Formats

    PPT, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document discusses neural networks and the Adaline algorithm. It provides an overview of the topics to be covered, including Perceptrons, calculating the Adaline algorithm for AND and XOR functions, and deriving the Least Mean Square (LMS) algorithm. The LMS algorithm is then explained in steps. Applications of Adaline are mentioned, including better convergence than Perceptrons and use in noise correction. Limitations are discussed, such as only working with linearly separable data, and how multi-level neural networks can address this.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PPT, PDF, TXT or read online from Scribd
    Download as ppt, pdf, or txt
    99 views14 pages

    NN Adaline

    Uploaded by

    Khasmad Ariansyah
    The document discusses neural networks and the Adaline algorithm. It provides an overview of the topics to be covered, including Perceptrons, calculating the Adaline algorithm for AND and XOR functions, and deriving the Least Mean Square (LMS) algorithm. The LMS algorithm is then explained in steps. Applications of Adaline are mentioned, including better convergence than Perceptrons and use in noise correction. Limitations are discussed, such as only working with linearly separable data, and how multi-level neural networks can address this.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PPT, PDF, TXT or read online from Scribd
    Download as ppt, pdf, or txt
    of 14

    Neural Networks - Adaline

    L. Manevitz

    NNs Adaline

    Plan of Lecture
    Perceptron: Connected and Convex examples Adaline Square Error Gradient Calculate for and, xor Discuss limitations LMS algorithm: derivation.
    NNs Adaline 2

    What is best weights?


    Most classified correctly? Least Square Error

    Least Square Error Before Cut-Off!


    To Minimize S (d Sw x)**2
    examples; second over dimension)
    (first sum over

    NNs Adaline

    Least Square Minimization


    Find gradient of error over all examples. Either calculate the minimum or move opposite to gradient.

    Widrow-Hoff(LMS): Use instantaneous example as approximation to gradient.


    Advantages: No memory; on-line; serves similar function as noise to avoid local problems. Adjust by w(new) = w(old) + a (d) x for each x. Here d = (desired output Swx)
    NNs Adaline 4

    LMS Derivation
    Errsq = S (d(k) W x(k)) ** 2 Grad(errsq) = 2S(d(k) W x(k)) (-x(k)) W (new) = W(old) - m Grad(errsq) To ease calculations, use Err(k) in place of Errsq

    W(new) = W(old) + 2mErr(k) x(k))


    Continue with next choice of k
    NNs Adaline 5

    Applications
    Adaline has better convergence properties than Perceptron

    Useful in noise correction


    Adaline in every modem.

    NNs Adaline

    LMS (Least Mean Square Alg.)


    1. Apply input to Adaline input 2. Find the square error of current input
    Errsq(k) = (d(k) - W x(k))**2

    3. Approximate Grad(ErrorSquare) by
    differentiating Errsq approximating average Errsq by Errsq(k) obtain -2Errsq(k)x(k)

    4.
    5.

    Update W: W(new) = W(old) + 2mErrsq(k)X(k)


    Repeat steps 1 to 4.

    NNs Adaline

    Comparison with Perceptron


    Both use updating rule changing with each input One fixes binary error; the other minimizes continuous error Adaline always converges; see what happens with XOR

    Both can REPRESENT Linearly separable functions


    NNs Adaline 8

    Convergence Phenomenom
    LMS converges depending on choice of m.

    How to choose it?

    NNs Adaline

    Limitations
    Linearly Separable How can we get around it?
    Use network of neurons?
    Use a transformation of data so that it is linearly separable

    NNs Adaline

    10

    Multi-level Neural Networks


    Representability
    Arbitrarily complicated decisions Continuous Approximation: Arbitrary Continuous Functions (and more) (Cybenko Theorem)

    Learnability
    Change Mc-P neurons to Sigmoid etc.

    Derive backprop using chain rule. (Like LMS TheoremSample Feed forward Network (No loops)

    NNs Adaline

    11

    Replacement of Threshold Neurons with Sigmoid or Differentiable Neurons

    Threshold
    NNs Adaline

    Sigmoid
    12

    Prediction
    delay Input/Output NN

    Compare

    NNs Adaline

    13

    Sample Feed forward Network (No loops)


    Weights Weights Weights

    Output

    Input

    Wji Vik

    F(S wji xj

    NNs Adaline

    14

    You might also like