Optimizing Plane Fitting Using Stochastic Gradient Descent

[zzmanzz]

Homework Statement

Suppose I wish to fit a plane
$$z = w_1 + w_2x +w_3y$$
to a data set $$(x_1,y_1,z_1), ... ,(x_n,y_n,z_n)$$

Using gradient descent

Homework Equations

http://en.wikipedia.org/wiki/Stochastic_gradient_descent

The Attempt at a Solution

I'm basically trying to figure out the 3-dimensional version of the example on wiki.
The objective function to e minimized is:

$$Q(w) = \sum_{i = 1}^n Q_i(w) = \sum_{i = 1}^n (w_1 + w_2x_i + w_3y_i - z_i)^2$$
I want to find the parameters of $$w_1,w_2,w_3$$

The iterative method updates the parameters $$w^{(0)}_1,w^{(0)}_2,w^{(0)}_3$$
1-step in the iteration
$$\left( \begin{array}{ccc} w^{(1)}_1 \\ w^{(1)}_2\\ w^{(1)}_3 \end{array} \right) = \left( \begin{array}{ccc} w^{(0)}_1 \\ w^{(0)}_2 \\ w^{(0)}_3 \end{array} \right) + \alpha \times \left( \begin{array}{ccc} 2(w^{(0)}_1 + w^{(0)}_2x_i + w^{(0)}_3 y_i - z_i) \\ 2x_i(w^{(0)}_1 + w^{(0)}_2x_i + w^{(0)}_3 y_i - z_i) \\ 2y_i(w^{(0)}_1+ w^{(0)}_2x_i + w^{(0)}_3 y_i - z_i) \end{array} \right)$$

[tex]\alpha [\tex] is the step size.

 

[Mugged]

Looks good. Whats your question?

 

[zzmanzz]

Just wanted to make sure that I didn't cheat or something in my solution. When I run it in c++ it works very well.