Scribd Logo
Upload

Welcome to Scribd!

  • 34 views

    Back Propogation PDF

    Uploaded by

    rajthakre81
    This document discusses different adaptive learning rate algorithms like Adagrad, SGD, Adadelta, Adam, and RMSProp that can be used for gradient descent optimization. It explains the difference between stochastic gradient descent and gradient descent, with stochastic gradient descent selecting random samples instead of the whole dataset for each iteration. The document also covers terminology related to gradient descent and discusses setting hyperparameters to prevent overfitting.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Back Propogation PDF

    Uploaded by

    rajthakre81
    34 views10 pages
    This document discusses different adaptive learning rate algorithms like Adagrad, SGD, Adadelta, Adam, and RMSProp that can be used for gradient descent optimization. It explains the difference between stochastic gradient descent and gradient descent, with stochastic gradient descent selecting random samples instead of the whole dataset for each iteration. The document also covers terminology related to gradient descent and discusses setting hyperparameters to prevent overfitting.

    Original Title

    Back Propogation.pdf

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document discusses different adaptive learning rate algorithms like Adagrad, SGD, Adadelta, Adam, and RMSProp that can be used for gradient descent optimization. It explains the difference between stochastic gradient descent and gradient descent, with stochastic gradient descent selecting random samples instead of the whole dataset for each iteration. The document also covers terminology related to gradient descent and discusses setting hyperparameters to prevent overfitting.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    34 views10 pages

    Back Propogation PDF

    Uploaded by

    rajthakre81
    This document discusses different adaptive learning rate algorithms like Adagrad, SGD, Adadelta, Adam, and RMSProp that can be used for gradient descent optimization. It explains the difference between stochastic gradient descent and gradient descent, with stochastic gradient descent selecting random samples instead of the whole dataset for each iteration. The document also covers terminology related to gradient descent and discusses setting hyperparameters to prevent overfitting.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 10

    SETTING THE LEARNING RATE

    ADAPTIVE LEARNING RATES


    ADAPTIVE LEARNING RATE ALGORITHMS

     Adagrad
     SGD
     Adadelta
     Adam
     RMS Prop
    GRADIENT DESCENT
    STOCHASTIC GRADIENT DESCENT

    The word ‘stochastic‘ means a system or a process


    that is linked with a random probability.
    Hence, in Stochastic Gradient Descent, a few
    samples are selected randomly instead of the whole
    data set for each iteration.

    In Gradient Descent, there is a term called “batch”


    which denotes the total number of samples from a
    dataset that is used for calculating the gradient for
    each iteration.
    TERMINOLOGY
    THREE MODES OF GRADIENT DESCENT
    SETTING HYPERPARAMETERS
    SETTING HYPERPARAMETERS
    THE PROBLEM OF OVERFITTING

    You might also like