Ridge Regression Minimization Proof

[zzmanzz]

Homework Statement

Linear family:

[tex]f(x;a) = a_{o} + (a_{1}.a_{2},a_{3},...,a_{k}) \cdot x[\tex]<br /> <br /> [tex](Xa - Y)^t \sigma^{-1} (Xa-Y) + \lambda (a^t a-a^2_{o} [\tex]<br /> <br /> [tex]a = (X^t \sigma^{-1} X + \lamda I_{o})^{-1} x^t \sigma^{-1} Y [\tex]<br /> <br /> <h2>Homework Equations</h2>[tex]Y_{i} = f(x_{i}) + N_{i} [\tex]<br /> <br /> i = 1,2,3,4,...,k.<br /> <br /> given data [tex]\{ (x_{i},y_{i})\}^2_{i} [\tex]<br /> <br /> [tex]x_{i} [\tex] = vector<br /> [tex]y_{i} [\tex] = scalar<br /> <br /> to minimize we write <br /> <br /> [tex](X(a+\Delta a) - Y)^t \sigma^{-1} (X(a+\Delta a)-Y)[\tex]<br /> <br /> [tex]=(Xa - Y)^t \sigma^{-1} (Xa-Y) + \Delta a^t X^t \sigma^{-1} (Xa-Y)+(Xa-Y)^t \sigma^{-1} + O(\Delta a^t \Delta a)[\tex]<br /> [tex]=(Xa - Y)^t \sigma^{-1} (Xa-Y) + 2\Delta a^t X^t \sigma^{-1} (Xa-Y) + O(\Delta a^t \Delta a)<br /> [\tex]<br /> cond for a:<br /> <br /> [tex]X^t \sigma^{-1} Xa - X^t \sigma^{-1} X = 0[\tex]<br /> <br /> $$a = (X^t \sigma^{-1} X)^(-1)X^t \sigma^{-1} Y[\tex]<br /> <br /> <h2>The Attempt at a Solution</h2><br /> <br /> Sorry can someone tell me why my latex is off? I am new to the forums and did my best to use the code? Thanks, I will repost.$$[/tex][/tex][/tex][/tex][/tex][/tex][/tex][/tex][/tex][/tex][/tex]

 

[Ray Vickson]

Homework Statement

Linear family:

[tex]f(x;a) = a_{o} + (a_{1}.a_{2},a_{3},...,a_{k}) \cdot x[\tex]<br /> <br /> [tex](Xa - Y)^t \sigma^{-1} (Xa-Y) + \lambda (a^t a-a^2_{o} [\tex]<br /> <br /> [tex]a = (X^t \sigma^{-1} X + \lamda I_{o})^{-1} x^t \sigma^{-1} Y [\tex]<br /> <br /> <h2>Homework Equations</h2><br /> <br /> <br /> [tex]Y_{i} = f(x_{i}) + N_{i} [\tex]<br /> <br /> i = 1,2,3,4,...,k.<br /> <br /> given data [tex]\{ (x_{i},y_{i})\}^2_{i} [\tex]<br /> <br /> [tex]x_{i} [\tex] = vector<br /> [tex]y_{i} [\tex] = scalar<br /> <br /> to minimize we write <br /> <br /> [tex](X(a+\Delta a) - Y)^t \sigma^{-1} (X(a+\Delta a)-Y)[\tex]<br /> <br /> [tex]=(Xa - Y)^t \sigma^{-1} (Xa-Y) + \Delta a^t X^t \sigma^{-1} (Xa-Y)+(Xa-Y)^t \sigma^{-1} + O(\Delta a^t \Delta a)[\tex]<br /> [tex]=(Xa - Y)^t \sigma^{-1} (Xa-Y) + 2\Delta a^t X^t \sigma^{-1} (Xa-Y) + O(\Delta a^t \Delta a)<br /> [\tex]<br /> cond for a:<br /> <br /> [tex]X^t \sigma^{-1} Xa - X^t \sigma^{-1} X = 0[\tex]<br /> <br /> $$a = (X^t \sigma^{-1} X)^(-1)X^t \sigma^{-1} Y[\tex]<br /> <br /> <h2>The Attempt at a Solution</h2><br /> <br /> Sorry can someone tell me why my latex is off? I am new to the forums and did my best to use the code? Thanks, I will repost.$$[/tex][/tex][/tex][/tex][/tex][/tex][/tex][/tex][/tex][/tex][/tex]

[tex][tex][tex][tex][tex][tex][tex][tex][tex][tex][tex]$$<br /> "\" starts a TeX/LaTeX command; "/" signals "end of TeX. That is, use "$$ instead of "[\tex]".<br /> <br /> RGV[/tex][/tex][/tex][/tex][/tex][/tex][/tex][/tex][/tex][/tex][/tex]