Finding the least square solution with only positive coefficients

[Marvin_MKII]

Hi everyone.

Is there any way to set demands on least square solutions?

I have an equation on the form Ax=b, which is solved for x as:

x=(A'*A)^-1*A'*b

I do know for a fact that all values in x should be positive, but the least square solution for my particular system contains a number of negative x-values.
Is there some way to find the least square solution with only positive coefficients?

Thanks in advance
Jocke

 

[conquest]

Hi Jocke,

It seems to me that since you have an explicit formula there for the least squares solution. Presumably because the rank of A equals the number of coefficients in x. You can only have one least squares solution.

So the only way to somehow get all positive entries for x may be to just perform a change of basis to make all the entries positive. Then also you would need to change A. This is if you look at the question in terms of spaces. If you just look at the problem in terms of a set of equations than it would seem one answer means one answer whether you agree with it or not.

So it may be that there is something about your problem I don't understand, it kind of seems like wanting something impossible.

 

[Marvin_MKII]

Thanks for the quick reply.

I think I have to expand my problem a bit.

What I have in b is a set of measured weights of products, and in A i have the amounts of different components used to create these weights, x is the component weights I want to find. In a smaller scale it could look something like this:
Axx=number of components, xx=component weight, bx =product weight

b1 = A11*x1+A12*x2+A13*x3
b2=A21*x1+A22*x2+A23*x3
...
bn=An1*x1+An2*x2+An3*x3

which in matrix-form would be:


\begin{bmatrix}
A11 & A12 & A13 \\
A21 & A22 & A23 \\
... & ... & ... \\
An1 & An2 & An3
\end{bmatrix}
\begin{bmatrix}
x1 \\
x2 \\
x3
\end{bmatrix} = \begin{bmatrix}
b1 \\
b2 \\
b3 \\
... \\
bn
\end{bmatrix}


To evaluate this method, I simulate values of x(say between 50 and 150) and combine them with A to get simulated b-values. Then I try to find the x-values again.
This approach works fine when I use simulated values of x without variation. When I simulate values of x with some variation, say ± 5%, I get some negative x-values in the results which it turn make other values way too high.
Since I'm dealing with simulated data I know that all values should be positive, and I want to find the "least square" with only positive x-values. I know there is no guarantee that this answer will be the correct one, but I still would like to try it.

Hope this clarifyed the problem a little bit.

Thanks again
Jocke

 

[SteamKing]

What sort of correlation are you getting with your fits? If r < 0.9, there may be other factors in play.

Also, forming normal equations can lead to round off error in certain situations.
For the ill-behaved data sets, you may wish to try a different method for least squares, say using the QR method to solve for the coefficients rather than forming the normal equations.

 

[Marvin_MKII]

I'm using Matlab to solve this problem and have tried

x=(A'A)^-1A'b
x=A\b
x=lsqnonneg(A,b)

The first two generates roughly the same answer, which contains negative values. The third one, which i found only yesterday, gives an answer with only positive or 0-values in x but it is still not even close to the "correct" one.

I was under the impression that A\b utilized the QR-method to some extent.

I have used Minitab to identify correlations between components, and removed one component of each pair with a correlation value of 1(always appear together). Do you think that high correlations amongst the remaining components could be a problem?

Am currently running a rather large lsqnonneg(A,b) in Matlab(A is 15541x1841 and b is 15541x1) so i haven't been able to check the correlations again.

Another possible problem might be that in the original data A was rank defficient, rank(A) was 1511 with 1845 columns.
To solve this I added 334 additional "measurements", each one containing only one of the components that lacked a pivot-element in rref(A) and nothing else.
(Rref(A) produces the row reduced echelon form of A)
Do you think that this approach might be problematic too?

I would be happy to explain the problem in further detail if needed, but I hope this is enough for now.Jocke