Optimizing Plane Fitting Using Stochastic Gradient Descent

[zzmanzz]

Homework Statement

Suppose I wish to fit a plane
z = w_1 + w_2x +w_3y
to a data set (x_1,y_1,z_1), ... ,(x_n,y_n,z_n)

Using gradient descent

Homework Equations

http://en.wikipedia.org/wiki/Stochastic_gradient_descent

The Attempt at a Solution

I'm basically trying to figure out the 3-dimensional version of the example on wiki.
The objective function to e minimized is:

Q(w) = \sum_{i = 1}^n Q_i(w) = \sum_{i = 1}^n (w_1 + w_2x_i + w_3y_i - z_i)^2
I want to find the parameters of w_1,w_2,w_3

The iterative method updates the parameters w^{(0)}_1,w^{(0)}_2,w^{(0)}_3
1-step in the iteration
<br /> \left( \begin{array}{ccc}<br /> w^{(1)}_1 \\<br /> w^{(1)}_2\\<br /> w^{(1)}_3 \end{array} \right) = \left( \begin{array}{ccc}<br /> w^{(0)}_1 \\<br /> w^{(0)}_2 \\<br /> w^{(0)}_3 \end{array} \right) + \alpha \times \left( \begin{array}{ccc}<br /> 2(w^{(0)}_1 + w^{(0)}_2x_i + w^{(0)}_3 y_i - z_i) \\<br /> 2x_i(w^{(0)}_1 + w^{(0)}_2x_i + w^{(0)}_3 y_i - z_i) \\<br /> 2y_i(w^{(0)}_1+ w^{(0)}_2x_i + w^{(0)}_3 y_i - z_i) \end{array} \right)

\alpha [\tex] is the step size.

 

[Mugged]

Looks good. Whats your question?

 

[zzmanzz]

Just wanted to make sure that I didn't cheat or something in my solution. When I run it in c++ it works very well.