1. PF Contest - Win "Conquering the Physics GRE" book! Click Here to Enter
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Ridge Regression Minimization Proof

  1. Jul 13, 2012 #1
    1. The problem statement, all variables and given/known data

    Linear family:

    [tex]f(x;a) = a_{o} + (a_{1}.a_{2},a_{3},...,a_{k}) \cdot x[\tex]

    [tex] (Xa - Y)^t \sigma^{-1} (Xa-Y) + \lambda (a^t a-a^2_{o} [\tex]

    [tex] a = (X^t \sigma^{-1} X + \lamda I_{o})^{-1} x^t \sigma^{-1} Y [\tex]

    2. Relevant equations

    [tex] Y_{i} = f(x_{i}) + N_{i} [\tex]

    i = 1,2,3,4,...,k.

    given data [tex] \{ (x_{i},y_{i})\}^2_{i} [\tex]

    [tex] x_{i} [\tex] = vector
    [tex] y_{i} [\tex] = scalar

    to minimize we write

    [tex](X(a+\Delta a) - Y)^t \sigma^{-1} (X(a+\Delta a)-Y)[\tex]

    [tex]=(Xa - Y)^t \sigma^{-1} (Xa-Y) + \Delta a^t X^t \sigma^{-1} (Xa-Y)+(Xa-Y)^t \sigma^{-1} + O(\Delta a^t \Delta a)[\tex]
    [tex]=(Xa - Y)^t \sigma^{-1} (Xa-Y) + 2\Delta a^t X^t \sigma^{-1} (Xa-Y) + O(\Delta a^t \Delta a)
    cond for a:

    [tex]X^t \sigma^{-1} Xa - X^t \sigma^{-1} X = 0[\tex]

    [tex]a = (X^t \sigma^{-1} X)^(-1)X^t \sigma^{-1} Y[\tex]

    3. The attempt at a solution

    Sorry can someone tell me why my latex is off? I am new to the forums and did my best to use the code? Thanks, I will repost.
    Last edited: Jul 13, 2012
  2. jcsd
  3. Jul 13, 2012 #2

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    "\" starts a TeX/LaTeX command; "/" signals "end of TeX. That is, use "[/tex] instead of "[\tex]".

Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Similar Threads - Ridge Regression Minimization Date
Conceptual Question regarding hypothesis testing regression Nov 10, 2017
Linear Least-Squares Regression: a_0 = ? Aug 1, 2017
Regression model Jun 5, 2016
Minimization and least squares/ridge regression Jul 18, 2012