Scribd Logo
Upload

Welcome to Scribd!

  • 22 views

    MLE MAP Regression PDF

    Uploaded by

    Alpha Wolf
    The document discusses parameter estimation and maximum likelihood estimation for regression problems. It defines the mean squared error cost function and derives the normal equations to find the optimal weight vector that minimizes this cost function. It also introduces maximum a posteriori estimation, where prior distributions over the parameters are included to address overfitting issues. The optimal parameters are estimated by maximizing the posterior distribution given the data.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    MLE MAP Regression PDF

    Uploaded by

    Alpha Wolf
    22 views11 pages
    The document discusses parameter estimation and maximum likelihood estimation for regression problems. It defines the mean squared error cost function and derives the normal equations to find the optimal weight vector that minimizes this cost function. It also introduces maximum a posteriori estimation, where prior distributions over the parameters are included to address overfitting issues. The optimal parameters are estimated by maximizing the posterior distribution given the data.

    Original Title

    MLE_MAP_Regression.pdf

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document discusses parameter estimation and maximum likelihood estimation for regression problems. It defines the mean squared error cost function and derives the normal equations to find the optimal weight vector that minimizes this cost function. It also introduces maximum a posteriori estimation, where prior distributions over the parameters are included to address overfitting issues. The optimal parameters are estimated by maximizing the posterior distribution given the data.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    22 views11 pages

    MLE MAP Regression PDF

    Uploaded by

    Alpha Wolf
    The document discusses parameter estimation and maximum likelihood estimation for regression problems. It defines the mean squared error cost function and derives the normal equations to find the optimal weight vector that minimizes this cost function. It also introduces maximum a posteriori estimation, where prior distributions over the parameters are included to address overfitting issues. The optimal parameters are estimated by maximizing the posterior distribution given the data.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 11

    Parameter estimation for

    regression

    -
    2 Regression : flashback

    –
    3

    –
    4
    ML estimation for
    the regression problem

    –
    5

    –
    6

    1 N
    E D ( w ) = ∑ (t n − yn ) 2
    2 n =1
    1
    = (t − Φ w )T ( t − Φ w )
    2
    1
    = (t − Φ w )T t − (t − Φ w )T Φ w
    2
    1 T
    = (t t − w T Φ T t − t T Φ w + w T Φ T Φ w )
    2

    Using w T Φ T t = t T Φw
    1 T
    = (t t − 2 w T Φ T t + w T Φ T Φ w )
    2
    –
    7

    ∂ED (w )
    = Φ T t − Φ T Φw
    ∂w
    At the optimal weight ∂ED (w ∗ )
    vector , we have =0
    ∂w

    ⇒ Φ T t − Φ T Φw ∗ = 0

    w ∗ = (Φ T Φ ) −1 Φ T t

    –
    8

    –
    9 Maximum a posterior estimation

    p (D | θ). p (θ)
    p (θ | D) =
    ∫ p(D | θ). p(θ)dθ
    N
    p ( D | θ) = ∏ p ( x k | θ)
    k =1

    θˆ = arg max θ
    p(θ | D)

    –
    10
    MAP estimation for
    the regression problem

    = + ln

    –
    11

    You might also like