1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: Stochastic Gradient Descent

  1. Mar 4, 2014 #1
    1. The problem statement, all variables and given/known data

    Suppose I wish to fit a plane
    [tex] z = w_1 + w_2x +w_3y [/tex]
    to a data set [tex] (x_1,y_1,z_1), ... ,(x_n,y_n,z_n) [/tex]

    Using gradient descent

    2. Relevant equations

    http://en.wikipedia.org/wiki/Stochastic_gradient_descent

    3. The attempt at a solution

    I'm basically trying to figure out the 3-dimensional version of the example on wiki.
    The objective function to e minimized is:

    [tex] Q(w) = \sum_{i = 1}^n Q_i(w) = \sum_{i = 1}^n (w_1 + w_2x_i + w_3y_i - z_i)^2 [/tex]
    I want to find the parameters of [tex]w_1,w_2,w_3 [/tex]

    The iterative method updates the parameters [tex]w^{(0)}_1,w^{(0)}_2,w^{(0)}_3 [/tex]
    1-step in the iteration
    [tex]
    \left( \begin{array}{ccc}
    w^{(1)}_1 \\
    w^{(1)}_2\\
    w^{(1)}_3 \end{array} \right) = \left( \begin{array}{ccc}
    w^{(0)}_1 \\
    w^{(0)}_2 \\
    w^{(0)}_3 \end{array} \right) + \alpha \times \left( \begin{array}{ccc}
    2(w^{(0)}_1 + w^{(0)}_2x_i + w^{(0)}_3 y_i - z_i) \\
    2x_i(w^{(0)}_1 + w^{(0)}_2x_i + w^{(0)}_3 y_i - z_i) \\
    2y_i(w^{(0)}_1+ w^{(0)}_2x_i + w^{(0)}_3 y_i - z_i) \end{array} \right) [/tex]

    [tex]\alpha [\tex] is the step size.
     
    Last edited: Mar 5, 2014
  2. jcsd
  3. Mar 8, 2014 #2
    Looks good. Whats your question?
     
  4. Mar 8, 2014 #3
    Just wanted to make sure that I didn't cheat or something in my solution. When I run it in c++ it works very well.
     
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted