# Solving for least square coefficients of power law

1. Oct 14, 2011

### enfield

$$y_i=A{x_i}^b$$

When I solve for A two different ways I am getting different answers..so somewhere I'm doing something wrong. If someone could point out where I would be grateful :).

Using logs:

$$y_i=A{x_i}^b$$
$$ln(y_i)=ln(A)+b*ln(x_i)$$
$$ln(y_i)-(ln(A)+b*ln(x_i))=r_i$$ for least squares we want to minimize: $$S=\sum_i^{n}{r_i}^2$$ which means the gradient has to be zero. I only care about finding A right now so I only have to deal with the partial with respect to A:
$$\frac{\partial S}{\partial A}\sum_i^{n}{r_i}^2 =2 \sum_i^{n}{r_i}\frac{\partial r_i}{\partial A}= 2\sum_i^{n}(ln(y_i)-ln(A)-b*ln(x_i))\frac{1}{A}=0$$

The numerator inside the sum has to be zero, and we can ignore the 2, so:

$$\sum_i^{n}(ln(y_i)-ln(A)-b*ln(x_i))=0$$
$$\sum_i^{n}ln(y_i)-n*ln(A)-b\sum_i^{n}ln(x_i)=0$$
$$ln(A)=\frac{\sum_i^{n}ln(y_i)-b\sum_i^{n}ln(x_i)}{n}$$

(this is the derivation that I think is correct).
But when i solve for A without taking the logs of each side first I get something else:

$$y_i - A{x_i}^b = r_i$$

$$\frac{\partial S}{\partial A}\sum_i^{n}{r_i}^2 =2 \sum_i^{n}{r_i}\frac{\partial r_i}{\partial A}= 2\sum_i^{n}(y_i - A{x_i}^b)*-{x_i}^b=0$$

$$\sum_i^{n}(-{x_i}{y_i} + A{x_i}^{2b})=0$$

$$-\sum_i^{n}{x_i}{y_i}+A\sum_i^{n}{x_i}^{2b}=0$$
$$A=\frac{\sum_i^{n}{x_i}{y_i}}{\sum_i^{n}{x_i}^{2b}}$$

And if you take the ln of it to compare it with what we got before you get:

$$ln(A)= ln(\frac{\sum_i^{n}{x_i}{y_i}}{\sum_i^{n}{x_i}^{2b}}) =ln(\sum_i^{n}{x_i}{y_i})-ln(\sum_i^{n}{x_i}^{2b})$$

Which is not the same as:

$$ln(A)=\frac{\sum_i^{n}ln(y_i)-b\sum_i^{n}ln(x_i)}{n}$$

as far as I can tell...

2. Oct 14, 2011

### hotvette

That's because you solved two different problems.

Minimizing $\sum[ln(y_i)-ln(A)-b*ln(x_i)]^2$ is not necessarily the same thing as minimizing $\sum(y_i - A{x_i}^b)^2$. The 2nd method will result in a lower residual unless the data happen to fit the curve exactly (i.e. zero residual), in which case both will yield the same result.

Last edited: Oct 14, 2011
3. Oct 14, 2011

### enfield

Ah, thanks so much. Yeah, I had thought they were maybe different problems, but I couldn't see exactly how.

When you do least squares with the log of the data and function you are fitting to the data you won't get the same coefficients for the function that you would if you just did least squares with the unmodified data/function.

Okay, that makes sense. Does lower residual mean lower mean squared error, so better fit?

4. Oct 14, 2011

### hotvette

.

Yes. Two different words for the same thing.