Scribd Logo
Upload

Welcome to Scribd!

  • 7 views

    Interpretatiof Logistic Regression

    Uploaded by

    Milton Espinoza
    The document describes using a logistic regression model to classify images from the MNIST dataset. The MNIST dataset contains images of handwritten digits that are encoded as numbers between 0-255. Logistic regression is used to build a predictive model that takes these encoded images as input and predicts the digit in the image.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Interpretatiof Logistic Regression

    Uploaded by

    Milton Espinoza
    7 views24 pages
    The document describes using a logistic regression model to classify images from the MNIST dataset. The MNIST dataset contains images of handwritten digits that are encoded as numbers between 0-255. Logistic regression is used to build a predictive model that takes these encoded images as input and predicts the digit in the image.

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document describes using a logistic regression model to classify images from the MNIST dataset. The MNIST dataset contains images of handwritten digits that are encoded as numbers between 0-255. Logistic regression is used to build a predictive model that takes these encoded images as input and predicts the digit in the image.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    7 views24 pages

    Interpretatiof Logistic Regression

    Uploaded by

    Milton Espinoza
    The document describes using a logistic regression model to classify images from the MNIST dataset. The MNIST dataset contains images of handwritten digits that are encoded as numbers between 0-255. Logistic regression is used to build a predictive model that takes these encoded images as input and predicts the digit in the image.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 24

    Interpretation of

    Logistic Regression
    MNIST Data Set

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Number
    0-9
    DATA MACHINE MODEL PREDICTION
    Images are Encoded as Numbers
    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 118 200 223 155 155 23 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 6 1248214 253 253 253 253 254 229 213 67 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 43 198 253 254 253 247 175 175 235 253 253 108 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 9 212 253 253 224 58 19 0 0 16 139 247 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 254 254 186 14 0 0 0 0 0 0 189 254 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 253 183 8 0 0 0 0 0 0 36 222 253 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 54 250 128 2 0 0 0 6 71 192 237 253 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 213 253 50 0 18 123 198 253 254 253 247 85 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 69 241 227 136 200 253 253 253 254 192 34 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 175 254 256 254 2545254 149 59 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 101 253 253 254 253 253 253 42 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 8 138 247 253 243 159 196 243 253 19 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 12 183 253 253 253 50 0 0 49 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 180 254 253 213 50 2 0 0 711 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 234 217 97 10 0 0 0 23 207 254 215 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 174 253 156 0 0 0 0 45 215 253 253 95 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 9 210 253 163 5 19 49 130 244 253 251 137 4 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 13 229 253 254 192 253 253 253 254 253 137 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 160 253 254 253 253 253 253 230 132 4 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 18 112 194 254 254 254 163 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Images are Encoded as Numbers
    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 118 200 223 155 155 23 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 6 1248214 253 253 253 253 254 229 213 67 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 43 198 253 254 253 247 175 175 235 253 253 108 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 9 212 253 253 224 58 19 0 0 16 139 247 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 254 254 186 14 0 0 0 0 0 0 189 254 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 253 183 8 0 0 0 0 0 0 36 222 253 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 54 250 128 2 0 0 0 6 71 192 237 253 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 213 253 50 0 18 123 198 253 254 253 247 85 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 69 241 227 136 200 253 253 253 254 192 34 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 175 254 256 254 2545254 149 59 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 101 253 253 254 253 253 253 42 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 8 138 247 253 243 159 196 243 253 19 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 12 183 253 253 253 50 0 0 49 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 180 254 253 213 50 2 0 0 711 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 234 217 97 10 0 0 0 23 207 254 215 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 174 253 156 0 0 0 0 45 215 253 253 95 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 9 210 253 163 5 19 49 130 244 253 251 137 4 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 13 229 253 254 192 253 253 253 254 253 137 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 160 253 254 253 253 253 253 230 132 4 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 18 112 194 254 254 254 163 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Images are Encoded as Numbers
    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 118 200 223 155 155 23 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 6 1248214 253 253 253 253 254 229 213 67 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 43 198 253 254 253 247 175 175 235 253 253 108 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 9 212 253 253 224 58 19 0 0 16 139 247 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 254 254 186 14 0 0 0 0 0 0 189 254 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 253 183 8 0 0 0 0 0 0 36 222 253 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 54 250 128 2 0 0 0 6 71 192 237 253 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 213 253 50 0 18 123 198 253 254 253 247 85 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 69 241 227 136 200 253 253 253 254 192 34 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 175 254 256 254 2545254 149 59 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 101 253 253 254 253 253 253 42 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 8 138 247 253 243 159 196 243 253 19 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 12 183 253 253 253 50 0 0 49 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 180 254 253 213 50 2 0 0 711 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 234 217 97 10 0 0 0 23 207 254 215 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 174 253 156 0 0 0 0 45 215 253 253 95 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 9 210 253 163 5 19 49 130 244 253 251 137 4 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 13 229 253 254 192 253 253 253 254 253 137 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 160 253 254 253 253 253 253 230 132 4 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 18 112 194 254 254 254 163 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Images are Encoded as Numbers
    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 118 200 223 155 155 23 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 6 1248214 253 253 253 253 254 229 213 67 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 43 198 253 254 253 247 175 175 235 253 253 108 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 9 212 253 253 224 58 19 0 0 16 139 247 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 254 254 186 14 0 0 0 0 0 0 189 254 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 253 183 8 0 0 0 0 0 0 36 222 253 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 54 250 128 2 0 0 0 6 71 192 237 253 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 213 253 50 0 18 123 198 253 254 253 247 85 0 0 0 0 0 0 0

    X=
    0 0 0 0 0 0 0 0 0 69 241 227 136 200 253 253 253 254 192 34 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 175 254 256 254 2545254 149 59 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 101 253 253 254 253 253 253 42 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 8 138 247 253 243 159 196 243 253 19 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 12 183 253 253 253 50 0 0 49 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 180 254 253 213 50 2 0 0 711 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 234 217 97 10 0 0 0 23 207 254 215 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 174 253 156 0 0 0 0 45 215 253 253 95 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 9 210 253 163 5 19 49 130 244 253 251 137 4 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 13 229 253 254 192 253 253 253 254 253 137 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 160 253 254 253 253 253 253 230 132 4 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 18 112 194 254 254 254 163 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
    Output is Binary

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Learning on MNIST
    Training Set Logistic Regression Model
    σ(zi) (or “Network”)
    zi = (b1 X xi1) + (b2 X xi2 (b2 X xi2) + ꔇ + (bM X xiM) + b0
    zi

    bM
    b1

    xi1 xiM

    (b0, b1,… bM)


    Learned Parameters
    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
    Zooming in on 0/1

    subject learned parameters


    – 0.2

    – 0.1

    σ( x – 0.0

    – –0.1
    )
    – –0.2

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Zooming in on 0/1

    – 0.2

    – 0.1

    σ( – 0.0

    – –0.1
    ) = 0.006
    – –0.2

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Zooming in on 0/1

    negative sections – 0.2

    – 0.1

    σ( – 0.0

    – –0.1
    ) = 0.006
    – –0.2

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Zooming in on 0/1

    – 0.2

    – 0.1

    σ( – 0.0

    – –0.1
    ) = 0.991
    – –0.2

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Zooming in on 0/1

    positive sections – 0.2

    – 0.1

    σ( – 0.0

    – –0.1
    ) = 0.991
    – –0.2

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Learned Weights for 0/1
    – 0.2

    – 0.1

    σ( x – 0.0

    – –0.1
    ) = 0.006
    (6% chance it is a 1)
    – –0.2

    – 0.2

    – 0.1

    σ( x – 0.0

    – –0.1
    ) = 0.991
    (99.1% chance it is a 1)
    – –0.2

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Interpretation of
    Logistic Regression
    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
    xi1 xi2 xiM
    add up the M products
    xi
    M

    b
    x x x
    Σ
    m=1
    xim X bm

    b1 b2 bM
    Interpretation of
    Logistic Regression
    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
    xi1 xi2 xiM
    add up the M products
    xi
    M

    b
    x x x
    Σ
    m=1
    xim X bm

    b1 b2 bM

    Compact Notation: xi ⊙b
    Interpretation of
    Logistic Regression
    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
    xi1 xi2 xiM
    add up the M products
    xi
    M

    b
    x x x
    Σ
    m=1
    xim X bm

    b1 b2 bM

    Compact Notation: xi ⊙b

    inner product
    Interpretation of Model

    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0


    = b0 + xi ⊙ b
    – 0.2

    – 0.1

    – 0.0
    examples of class y = 0
    – –0.1

    – –0.2

    filter
    examples of class y = 1
    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
    Interpretation of Model

    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0


    = b0 + xi ⊙ b
    – 0.2

    – 0.1

    – 0.0
    examples of class y = 0
    – –0.1

    – –0.2

    filter
    examples of class y = 1
    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
    Interpretation of Model

    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0


    = b0 + xi ⊙ b
    – 0.2

    – 0.1

    – 0.0
    examples of class y = 0
    – –0.1

    – –0.2

    filter b
    examples of class y = 1
    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
    Interpretation of Model

    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0


    Outcome of y
    = b0 + xi ⊙ b
    Strong match between filter
    – 0.2
    and data indicates p(yi = 1|xi)
    is high
    – 0.1
    Low match between filter
    – 0.0 and data indicates p(yi = 1|xi)
    examples of class y = 0 is low
    – –0.1

    – –0.2

    filter b
    examples of class y = 1
    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
    Interpretation of
    Logistic Regression
    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
    Sigmoid Function p(yi = 1|xi) = σ(zi)
    1
    – 0.2

    – 0.1

    – 0.0
    0.5
    – –0.1

    – –0.2

    –6
    0 zi filter b
    –4 –2 0 2 4 6

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Credits
    MNIST Dataset of Handwritten Digits (Images)
    Yann LeCun (Courant Institue, NYU) and Corinna Cortes (Google Labs, New York) CC-by-SA 3.0
    http://yann.lecun.com/exdb/mnist/

    You might also like