1. Not finding help here? Sign up for a free 30min tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Ridge Regression Minimization Proof

  1. Jul 13, 2012 #1
    1. The problem statement, all variables and given/known data

    Linear family:

    [tex]f(x;a) = a_{o} + (a_{1}.a_{2},a_{3},...,a_{k}) \cdot x[\tex]

    [tex] (Xa - Y)^t \sigma^{-1} (Xa-Y) + \lambda (a^t a-a^2_{o} [\tex]

    [tex] a = (X^t \sigma^{-1} X + \lamda I_{o})^{-1} x^t \sigma^{-1} Y [\tex]

    2. Relevant equations


    [tex] Y_{i} = f(x_{i}) + N_{i} [\tex]

    i = 1,2,3,4,...,k.

    given data [tex] \{ (x_{i},y_{i})\}^2_{i} [\tex]

    [tex] x_{i} [\tex] = vector
    [tex] y_{i} [\tex] = scalar

    to minimize we write

    [tex](X(a+\Delta a) - Y)^t \sigma^{-1} (X(a+\Delta a)-Y)[\tex]

    [tex]=(Xa - Y)^t \sigma^{-1} (Xa-Y) + \Delta a^t X^t \sigma^{-1} (Xa-Y)+(Xa-Y)^t \sigma^{-1} + O(\Delta a^t \Delta a)[\tex]
    [tex]=(Xa - Y)^t \sigma^{-1} (Xa-Y) + 2\Delta a^t X^t \sigma^{-1} (Xa-Y) + O(\Delta a^t \Delta a)
    [\tex]
    cond for a:

    [tex]X^t \sigma^{-1} Xa - X^t \sigma^{-1} X = 0[\tex]

    [tex]a = (X^t \sigma^{-1} X)^(-1)X^t \sigma^{-1} Y[\tex]

    3. The attempt at a solution

    Sorry can someone tell me why my latex is off? I am new to the forums and did my best to use the code? Thanks, I will repost.
     
    Last edited: Jul 13, 2012
  2. jcsd
  3. Jul 13, 2012 #2

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    "\" starts a TeX/LaTeX command; "/" signals "end of TeX. That is, use "[/tex] instead of "[\tex]".

    RGV
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Ridge Regression Minimization Proof
  1. Logistic Regression (Replies: 1)

Loading...