I Decomposing Vectors Using Row Reduction: A Practical Approach

[dman12]

Hello,

I am trying to figure out how to best decompose a vector into a best fit linear superposition of other, given vectors.

For instance is there a way of finding the best linear sum of:

(3,5,7,0,1)
(0,0,4,5,7)
(8,9,2,0,4)

That most closely gives you (1,2,3,4,5)

My problem contains more, higher order vectors so if there is a general statistical way of doing a decomposition like this that would be great.

Thanks!

 

[blue_leaf77]

You can use least square solution. First, realize that you can express a linear combination of $n$ $m\times 1$ column vectors as a matrix product between a matrix formed by placing those $n$ columns next to each other and a $n \times 1$ column vector consisting of the coefficients of each vector in the sum. Denote the first matrix as $A$ and the second (column) one as $x$, you are to find $x$ such that $||Ax-b||$ is minimized where $b$ is the $m \times 1$ column vector you want to fit to.

 

[BvU]

My hunch was that the three vectors span a 3D space in which you can express the part of (1,2,3,4,5) that lies in that space exactly (by projections). For the two other dimensions there's nothing you can do. Am I deceiving myself ?

 

[chiro]

Hey dman12.

This is equivalent to solving the linear system in RREF.

Understanding this process of row reduction and why it works will help you understand a lot of linear algebra in a practical capacity.