1. Not finding help here? Sign up for a free 30min tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Stochastic Gradient Descent

  1. Mar 4, 2014 #1
    1. The problem statement, all variables and given/known data

    Suppose I wish to fit a plane
    [tex] z = w_1 + w_2x +w_3y [/tex]
    to a data set [tex] (x_1,y_1,z_1), ... ,(x_n,y_n,z_n) [/tex]

    Using gradient descent

    2. Relevant equations

    http://en.wikipedia.org/wiki/Stochastic_gradient_descent

    3. The attempt at a solution

    I'm basically trying to figure out the 3-dimensional version of the example on wiki.
    The objective function to e minimized is:

    [tex] Q(w) = \sum_{i = 1}^n Q_i(w) = \sum_{i = 1}^n (w_1 + w_2x_i + w_3y_i - z_i)^2 [/tex]
    I want to find the parameters of [tex]w_1,w_2,w_3 [/tex]

    The iterative method updates the parameters [tex]w^{(0)}_1,w^{(0)}_2,w^{(0)}_3 [/tex]
    1-step in the iteration
    [tex]
    \left( \begin{array}{ccc}
    w^{(1)}_1 \\
    w^{(1)}_2\\
    w^{(1)}_3 \end{array} \right) = \left( \begin{array}{ccc}
    w^{(0)}_1 \\
    w^{(0)}_2 \\
    w^{(0)}_3 \end{array} \right) + \alpha \times \left( \begin{array}{ccc}
    2(w^{(0)}_1 + w^{(0)}_2x_i + w^{(0)}_3 y_i - z_i) \\
    2x_i(w^{(0)}_1 + w^{(0)}_2x_i + w^{(0)}_3 y_i - z_i) \\
    2y_i(w^{(0)}_1+ w^{(0)}_2x_i + w^{(0)}_3 y_i - z_i) \end{array} \right) [/tex]

    [tex]\alpha [\tex] is the step size.
     
    Last edited: Mar 5, 2014
  2. jcsd
  3. Mar 8, 2014 #2
    Looks good. Whats your question?
     
  4. Mar 8, 2014 #3
    Just wanted to make sure that I didn't cheat or something in my solution. When I run it in c++ it works very well.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted



Similar Discussions: Stochastic Gradient Descent
  1. Stochastic processes (Replies: 14)

  2. Stochastic matrix (Replies: 0)

  3. Stochastic Process (Replies: 0)

  4. Stochastic Processes (Replies: 1)

Loading...