Linear AlgebraConceptual Question

[Saladsamurai]

I was watching the MIT opencourseware lecture #1 by Gilbert Strang last night. He attempts to introduce two different ideas in this lecture.

The first is that a system of equations, let's use 2 eqs & 2 unknowns for example, can be viewed from a row picture. This is the way I am used to looking at it. Each row represents the EQ of a line and where those two lines intersect is the solution to the system.

The second is that the system can be viewed from the column picture. That is if we have the following system:

$$\begin{array}{c}a_1x+b_1y=c_1 \\ a_2x+b_2y=c_2\end{array}$$

we can look at each column as a vector like this:

$$x\left[\begin{array}{c}a_1\\a_2\end{array}\right]+y\left[\begin{array}{c}b_1\\b_2\end{array}\right]=\left[\begin{array}{c}c_1\\c_2\end{array}\right]$$

And in doing so, we now seek to ask what the proper multiples of the column vectors are such that when added, they yield the correct answer.

He then drew it out, that is he showed how it worked geometrically.

I am a little confused. I can see that it does indeed work. But I do not see why? Why should the columns be treated as vectors? How are they related? They seem to me to be completely separate entities... How did one discover that this method will work?It is hard to put my question into words, so if I am being to vague or unclear, please yell at me.

 

[Mark44]

The quick answer is that both ways produce exactly the same set of equations. Strang might be leading up to yet another way to look at the system of two equations, in which the left side is written as a product of a 2 x 2 matrix and a column vector. Notice that the columns of the 2 x 2 matrix are the same as the columns in his column picture.

$$\left[\begin{array}{cc}a_1 &b_1\\a_2 &b_2\end{array}\right]\left[\begin{array}{c}x \\y\end{array}\right]=\left[\begin{array}{c}c_1\\c_2\end{array}\right]$$

In answer to some of your questions, the columns are unrelated to each other except that each column has two entries. I don't know what else to say other than what Strang has presented is just another way of looking at things.

 

[Saladsamurai]

Well... I just do not see why one should consider the columns vectors?

He literally draws a coordinate plane and then sketches the vectors <a1,a2> and <b1,b2> and then draws multiples of them until they add to <c1,c2>

It's just weird to me. Why should <a1,a2> even be a vector?

 

[Dick]

Mark44 already said it but look at the components of the matrix equation vs the equation x*[a1,a2]+y*[b1,b2]=[c1,c2]. They are the same. You are right, [a1,a2] isn't a vector, it's a column of a matrix. In math language, the two problems are isomorphic. A solution of one is a solution of the other (once you exchange 'column of a matrix' for 'vector') and vice versa. So it doesn't matter much what they REALLY are. If you solve one, you solve the other.

 

[Saladsamurai]

Well I guess I just have to deal with it then. I just don't understand why if you actually draw out the 'vectors' it works geometrically.

Like if I have:

$$x\left[\begin{array}{c}1\\2\end{array}\right]+y\left[\begin{array}{c}2\\4\end{array}\right]=\left[\begin{array}{c}4\\8\end{array}\right]$$

If I draw out 2*<1,2> + 1*<2,4> it will lead to <4,8>

Perhaps I should just get over it for now and maybe it will come with time.

Also Dick: Do we need to have a talk? Is everything okay? Your post count has gotten a little ridiculous!

 

[dx]

I just don't understand why if you actually draw out the 'vectors' it works geometrically.

They are only 'vectors' in the sense that they are elements of the space R², i.e. the set of objects of the form (a,b) where a and b are real numbers. There's no connection here with the usual vectors of elementary physics like displacement vectors in euclidean space.

 

[Saladsamurai]

They are only 'vectors' in the sense that they are elements of the space R², i.e. the set of objects of the form (a,b) where a and b are real numbers. There's no connection here with the usual vectors of elementary physics like displacement vectors in euclidean space.

I understand that. But, the fact still remains that when you draw them out as displacement vectors they do indeed generate the solution geometrically. For example, the two 'vectors' in post #5

If you start at the origin and draw 2*<1,2> + <2,4> it will point to <4,8>

But like I said, I am just going to let it rest for awhile.

 

[Mark44]

They are only 'vectors' in the sense that they are elements of the space R², i.e. the set of objects of the form (a,b) where a and b are real numbers. There's no connection here with the usual vectors of elementary physics like displacement vectors in euclidean space.

The space R², with the usual operations for addition and multiplication by a scalar is a vector space. This vector space, like all vector spaces, adheres to the ten axioms involving addition of elements of the vector space and multiplication of elements in the vector space by scalars. The vectors used in physics seem to me to be identical to vectors in R² and R³, so I'm surprised to see that you say there's no connection.

 

[Knissp]

Another way to look at it would be in terms of components.

x + 2y = 4
2x + 4y = 8

x=2, y=1

"If I draw out 2*<1,2> + 1*<2,4> it will lead to <4,8>"

Look at the x-components of the vectors. It is easy to see how they are the same as the first equation. x+2y = 4 == 1*1 + 2*1 = 4
Then look at the y-components of the vectors, which matches up with the second equation. 2x+4y = 8 == 2*2 + 4*1 = 8

Now, the "column picture" is really just composed of the first equation as the x-component of the vector, and the second equation as the y-component of the vector. Since a vector is just the sum of its components, if you have a working equation for each of the two components, then you have a working vector picture.

I would like to point out that the x-and-y-components of the vector are not the same as the x and y that were used in the equation. On one hand, x-and-y refer to the coordinate system where you have two simultaneously valid equations in mutually perpendicular directions that you combine to form a convenient picture; on the other, you have x and y referring to particular points where the equalities are met, i.e. x=2, y=1.

 

[dx]

The space R², with the usual operations for addition and multiplication by a scalar is a vector space. This vector space, like all vector spaces, adheres to the ten axioms involving addition of elements of the vector space and multiplication of elements in the vector space by scalars. The vectors used in physics seem to me to be identical to vectors in R² and R³, so I'm surprised to see that you say there's no connection.

The vectors of physics like displacement vectors are naturally thought of as elements of the Euclidean vector space E³, which has more structure than R³. The OP seems to be surprised that the graphical method of interpreting the columns as vectors works, but the set of columns of length 2 with the rule of component-wise addition is clearly isomorphic to R², so I thought maybe he was not familiar with the usage of the word 'vector' in the sense of being an element of an arbitrary vector space, and was still thinking of Euclidean vectors.

 

[Дьявол]

Can I ask you also a linear algebra::conceptual question?

For ex. $\mathbb{R}^3$ is vector space defined with the standard addition and scalar multiplication. The vector (x,y,z) $\in \mathbb{R}^3$. But if we take an example, all vectors in the form $(1,x_1,y_1)$ is not vector subspace of $\mathbb{R}^3$ since it satisfies none of the standard addition or scalar multiplication.

But if we said that the vectors of the form (x,y,z) are vector space, why x=1, (1,y,z) is not?

Thanks in advance.

Regards.

 

[dx]

But if we said that the vectors of the form (x,y,z) are vector space, why x=1, (1,y,z) is not?

Elements of the form (1,y,z) certainly are vectors (elements of R³), but they do not constitute a vector subspace of R³.

 

[HallsofIvy]

The set of all vectors in R³, of the form (1, y, z) is not a subspace of R³ because it is not closed under vector addition- (1, a, b)+ (1, c, d)= (2, a+c, b+ d) is not of that form; it doesn't have "1" as first member- or under scalar multiplication- 3(1, a, b)= (3, 3a, 3b) is not of that form.

 

[Дьявол]

Yes, I know that it isn't subspace, but it is pretty strange, since the vectors (x,y,z) constitute the vector space R³, and the set of all vectors with the form (1,y,z) is not subspace. That means that all vectors (1,1,2), (1,3,4), (1,3,5) ... are not in R³.

I imagine it, like this:

 

[dx]

Yes, I know that it isn't subspace, but it is pretty strange, since the vectors (x,y,z) constitute the vector space R³, and the set of all vectors with the form (1,y,z) is not subspace. That means that all vectors (1,1,2), (1,3,4), (1,3,5) ... are not in R³.

You are confusing 'subspace' and 'subset'; they are different things. The set of elements of the form (1,y,z) is a subset of R³, but it is not a subspace.

 

[Mark44]

The vectors of physics like displacement vectors are naturally thought of as elements of the Euclidean vector space E³, which has more structure than R³. The OP seems to be surprised that the graphical method of interpreting the columns as vectors works, but the set of columns of length 2 with the rule of component-wise addition is clearly isomorphic to R², so I thought maybe he was not familiar with the usage of the word 'vector' in the sense of being an element of an arbitrary vector space, and was still thinking of Euclidean vectors.

As far as I know, there's no difference between what you're calling E³ and R³, unless you're thinking that the cross product is this extra structure.

 

[dx]

As far as I know, there's no difference between what you're calling E³ and R³, unless you're thinking that the cross product is this extra structure.

E³ is just R³ equipped with a positive definite symmetric bilinear form (scalar product).

 

[Дьявол]

You are confusing 'subspace' and 'subset'; they are different things. The set of elements of the form (1,y,z) is a subset of R³, but it is not a subspace.

dx you are right. Now I read the definition of subspace again, and realized that I misunderstood it.

I have one more question.

The vector space $\mathbb{R}^n$ satisfies all axioms of vector space. But why I don't understand it, the first ones standard addition and scalar multiplication.

For ex. $u+v \in \mathbb{R}^n$ and $cu \in \mathbb{R}^n$

$$u+v=(x_1,x_2,...,x_n) +(y_1,y_2,...,y_n)=(x_1+y_1,x_2+y_2,...,x_n+y_n)$$

$$cu=c(x_1,x_2,...,x_n)=(cx_1,cx_2,...,cx_n)$$

Now, how do we know that the standard addition and scalar multiplication are back in $\mathbb{R}^n$ ??

Thanks in advance.

 

[HallsofIvy]

dx you are right. Now I read the definition of subspace again, and realized that I misunderstood it.

I have one more question.

The vector space $\mathbb{R}^n$ satisfies all axioms of vector space. But why I don't understand it, the first ones standard addition and scalar multiplication.

For ex. $u+v \in \mathbb{R}^n$ and $cu \in \mathbb{R}^n$

$$u+v=(x_1,x_2,...,x_n) +(y_1,y_2,...,y_n)=(x_1+y_1,x_2+y_2,...,x_n+y_n)$$

$$cu=c(x_1,x_2,...,x_n)=(cx_1,cx_2,...,cx_n)$$

Now, how do we know that the standard addition and scalar multiplication are back in $\mathbb{R}^n$ ??

Thanks in advance.

"Back in $\mathbb{R}^n$"? Do you that $\mathbb{R}^n$ is closed under this addition and scalar multiplication? That is true because $\mathbb{R}$ itself is closed under addition and multiplication. Since $\mathbb{R}$ is closed under scalar multiplication, each of $x_1+ y_1$, $x_2+ y_2$, ..., $x_n+ y_n$ is in $\mathbb{R}$ and so [itex](x_1+y_1,x_2+y_2,...,x_n+y_n)[/tex] is an ordered n-tuple of number in $\mathbb{R}$, i.e. it is in $\mathbb{R}^n$. Since $\mathbb{R}$ is closed under multiplication, each of $cx_1$, $cx_2$, ..., $cx_n$ is in $\mathbb{R}$ and so $$cu=c(x_1,x_2,...,x_n)=(cx_1,cx_2,...,cx_n)$$ is an ordered n-tuple of real numbers, i.e., it is in $\mathbb{R}$.

 

[Дьявол]

But what is the proof? I am interested in the proof.

 

[Mark44]

E³ is just R³ equipped with a positive definite symmetric bilinear form (scalar product).

Can you elaborate?

 

[dx]

Can you elaborate?

Sure.

A bilinear form on a real vector space V is a map

$$\tau : V \times V \rightarrow \mathbb{R}$$

which is linear in both arguments. It is symmetric if τ(u,v) = τ(v,u) for all v and u in V. It is positive definite if τ(v,v) > 0 for all v ≠ 0.

 

[HallsofIvy]

But what is the proof? I am interested in the proof.

What I gave was the proof.