1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: Ridge Regression Minimization Proof

  1. Jul 13, 2012 #1
    1. The problem statement, all variables and given/known data

    Linear family:

    [tex]f(x;a) = a_{o} + (a_{1}.a_{2},a_{3},...,a_{k}) \cdot x[\tex]

    [tex] (Xa - Y)^t \sigma^{-1} (Xa-Y) + \lambda (a^t a-a^2_{o} [\tex]

    [tex] a = (X^t \sigma^{-1} X + \lamda I_{o})^{-1} x^t \sigma^{-1} Y [\tex]

    2. Relevant equations

    [tex] Y_{i} = f(x_{i}) + N_{i} [\tex]

    i = 1,2,3,4,...,k.

    given data [tex] \{ (x_{i},y_{i})\}^2_{i} [\tex]

    [tex] x_{i} [\tex] = vector
    [tex] y_{i} [\tex] = scalar

    to minimize we write

    [tex](X(a+\Delta a) - Y)^t \sigma^{-1} (X(a+\Delta a)-Y)[\tex]

    [tex]=(Xa - Y)^t \sigma^{-1} (Xa-Y) + \Delta a^t X^t \sigma^{-1} (Xa-Y)+(Xa-Y)^t \sigma^{-1} + O(\Delta a^t \Delta a)[\tex]
    [tex]=(Xa - Y)^t \sigma^{-1} (Xa-Y) + 2\Delta a^t X^t \sigma^{-1} (Xa-Y) + O(\Delta a^t \Delta a)
    cond for a:

    [tex]X^t \sigma^{-1} Xa - X^t \sigma^{-1} X = 0[\tex]

    [tex]a = (X^t \sigma^{-1} X)^(-1)X^t \sigma^{-1} Y[\tex]

    3. The attempt at a solution

    Sorry can someone tell me why my latex is off? I am new to the forums and did my best to use the code? Thanks, I will repost.
    Last edited: Jul 13, 2012
  2. jcsd
  3. Jul 13, 2012 #2

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    "\" starts a TeX/LaTeX command; "/" signals "end of TeX. That is, use "[/tex] instead of "[\tex]".

Share this great discussion with others via Reddit, Google+, Twitter, or Facebook