Scribd Logo
Upload

Welcome to Scribd!

  • 28 views

    Convolutional Neural Networks (1) : Geena Kim

    Uploaded by

    Huston LAM
    Convolutional neural networks use convolutional layers that apply filters to images to learn features. CNNs are effective for images because they incorporate properties like translational invariance and shared weights to reduce parameters. Convolutional layers are followed by pooling layers that reduce spatial dimensions. Multiple convolution layers allow the model to learn higher level features like shapes from lower level features like lines and colors. CNNs are trained end-to-end using backpropagation and gradient descent.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Convolutional Neural Networks (1) : Geena Kim

    Uploaded by

    Huston LAM
    28 views28 pages
    Convolutional neural networks use convolutional layers that apply filters to images to learn features. CNNs are effective for images because they incorporate properties like translational invariance and shared weights to reduce parameters. Convolutional layers are followed by pooling layers that reduce spatial dimensions. Multiple convolution layers allow the model to learn higher level features like shapes from lower level features like lines and colors. CNNs are trained end-to-end using backpropagation and gradient descent.

    Original Title

    4_C3_M3_L1-ConvolutionalNeuralNetwork-intro

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Convolutional neural networks use convolutional layers that apply filters to images to learn features. CNNs are effective for images because they incorporate properties like translational invariance and shared weights to reduce parameters. Convolutional layers are followed by pooling layers that reduce spatial dimensions. Multiple convolution layers allow the model to learn higher level features like shapes from lower level features like lines and colors. CNNs are trained end-to-end using backpropagation and gradient descent.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    28 views28 pages

    Convolutional Neural Networks (1) : Geena Kim

    Uploaded by

    Huston LAM
    Convolutional neural networks use convolutional layers that apply filters to images to learn features. CNNs are effective for images because they incorporate properties like translational invariance and shared weights to reduce parameters. Convolutional layers are followed by pooling layers that reduce spatial dimensions. Multiple convolution layers allow the model to learn higher level features like shapes from lower level features like lines and colors. CNNs are trained end-to-end using backpropagation and gradient descent.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 28

    Convolutional Neural Networks (1)

    Geena Kim
    Why Deep Learning?

    https://www.youtube.com/watch?v=EhcpGpFHCrw
    Why Deep Learning?
    Medical Imaging,
    AI in Medicine

    P. Rajpurkar et el.,
    Why Deep Learning?

    Shopping,
    e-commerce
    Success of deep learning
    Why Deep Learning?
    Voice
    Recog
    Smar
    t Dev nition,
    ices
    What is Artificial Neuron (Perceptron)
    Activation functions
    • Binary Threshold (Step function) • Sigmoid

    • Tanh
    • Softmax
    Activation functions
    • Sigmoid • Softmax

    OR
    Activation functions

    • Rectified Linear (ReLU) - Better convergence


    - Does not saturate
    - Less computation

    • Parametric Rectified Linear (PReLU) or Leaky ReLU


    Multi-layer Perceptron (Neural network)

    Design Parameters

    - Architecture
    - Number of layers
    - Number of neurons in a layer
    - Activation functions
    How Neural Network Training Works

    hj = f(wij * Xi + bj)
    j k
    Weight update rule
    i (Gradient descent)
    0
    y_pred y_target
    1
    …
    2

    wij wjk wk Chain rule


    error/loss

    input hidden hidden output


    X_train
    layer layer 1 layer 2 layer
    Dealing with images
    An image is a multi-dimension array

    Depth Width
    or
    Channel

    Height
    Dealing with images- filters
    Gaussian Blurring Mexican hat filtering
    What is Convolution (2D) in an Image?

    filter

    Image
    Convolutional Layer
    Q1. What is the feature map dimension, after a convolution layer with N filters?

    2D Convolution
    Convolutional Neural Network

    W
    W W

    FC FC out
    Conv Conv Conv

    W W W
    Convolutional Layer- hyper parameters
    • Filter size (3x3, 5x5, • Padding • Stride
    1x1….)

    • Number of filters

    ** Parameters and Hyper parameters are different!


    Parameters = Weights to be optimized.
    Hyperparameters = design parameters you can control
    Why CNN? What do the convolution filters do?

    • Images have big pixels!

    • Fully-connected neural network would have too many parameters!

    • Translational invariance in images


    Why CNN? What do the convolution filters do?

    h’

    w’
    d

    - Learning Filters
    - Weight Sharing
    - Computational Efficiency
    - Translational Invariance
    - Robust: less overfit
    Typical CNN architecture

    [ conv conv maxpool


    ] n
    convolutions with small filter size Dense out
    What is a Pooling layer?

    • Pooling is like sub-sampling


    • Pooling filter size usually is 2x2 (or 2n x 2n)
    • Usually reduce the size to 1/N per each side (e.g. N=2 for 2x2)
    • Max Pool
    • Average Pool
    Max pooling operation example
    Another way to shrink the spatial dimension
    We can reduce the spatial dimension using convolution filters with a bigger stride

    • Str
    id
    e
    What do the multiple convolution layers do?
    What do the multiple convolution layers do?
    Geometrical shapes are made of pieces of small curves, lines, and color blobs
    What do the multiple convolution layers do?

    arXiv:1311.2901v3
    Next time!

    • CNN Architectures
    • How to train CNNs

    You might also like