Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Linear Independence/Dependence

  1. Dec 2, 2012 #1

    I am new to the forums. I will try and keep this short.

    Linear Independence vs Linear Dependence. It's easier to understand from a vector perspective but when it's brought back to a system of equations, I get twisted in symantics.

    (1) Consistent: Linearly independent vs Linearly dependent. If it's consistent, it has one or more solutions: Linear independent (one solution). Linear dependent (infinite solutions). Linear independent also means equations can't be expressed as a linear combination of the others. If they are Linear dependent, they can.

    (2) Inconsistent: Linearly independent vs Linearly dependent. Since inconsistent means no solutions. You can't label a system of equations as linearly independent/dependent since both means a solution exists.

    I thought I had all this straight until I read a paper here. This gentleman quotes James/James, Mathematics Directory as his source and doesn't seem to agree with (2).

    If anyone can untangle me, I would appreciate it.

  2. jcsd
  3. Dec 2, 2012 #2
    Yes you got the right ideas going for linearly independence and linearly dependence. A better way to put the second part is the only time a linear system is inconsistent is when the column vector b becomes a pivot column after you have row reduced the matrix. In other wards when get a row that looks like this. (0,0...0|b). Thus the linear system is neither linearly indep. or linearly dependent. I looked at the paper and how he words his theorem is correct but confusing.
  4. Dec 2, 2012 #3
    Linear independence/dependence is a property of vectors and matrices, not of equations.

  5. Dec 2, 2012 #4
    Why do you think equations, or even just two numbers cannot be linearly dependent?

    Formally two sets of elements are linearly dependent if there exists a linear combination of these elements equal to zero, where the coefficients of the combination are not all themselves zero.

    Two sets are linearly independent if they are not dependent.

    For example the sets {1, ∏} and {∏, ∏2} are linearly dependent, if we can draw coefficients from R but not (ie linearly independent) if the coefficients have to come from Q.
  6. Dec 2, 2012 #5
    Those two sets are linearly dependent since one element (π) of the first set is equal to a linear combination (1π+0π2) of the elements of the second set. And the coefficients 0 and 1 are members of any field, not just of R.
  7. Dec 2, 2012 #6
    What coefficient in Q can you multiply the first set by to obtain the second?

    At least we are agreed that the sets are R-linearly dependent.
  8. Dec 2, 2012 #7


    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    I'm not sure what you're doing here. It looks like you have misinterpreteted some definition.

    This is the definition of "linearly independent" in the context of finite-dimensional vector spaces:

    Let V be a finite-dimensional vector space over a field ##\mathbb F##. A set ##\{x_1,\dots,x_n\}\subset V## is said to be linearly independent, if for all ##(a_1,\dots,a_n)\in\mathbb F^n## such that
    $$a_1x_1+\dots+a_nx_n=0,$$ we have ##a_1=\cdots=a_n=0##.

    Note that it's always one set that's linearly independent or linearly dependent, not two.
    Last edited: Dec 2, 2012
  9. Dec 2, 2012 #8
    Hello Fredrik,

    You might like to review your penultimate line ?

    Where is it decreed that linear dependence must be viewed in terms of finite dimensional edit: vector spaces?

    Yes, strictly the elements that the adjective 'linear dependence' applies to may be gathered together into one set.

    Provided that there is more than one element!

    However the elements of the set may themselves be sets, which is the case in my example.

    The elements may also be equations, which was disputed by Bipolarity and the subject of the original question and the reason for my first post in this thread.
    Last edited: Dec 2, 2012
  10. Dec 2, 2012 #9
    The elements could certainly be functions, but it would be very unusual to have a set of equations as a vector space (as far as I know).
  11. Dec 2, 2012 #10


    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    Hi Studiot. I don't see anything wrong with the penultimate line (unless you just meant that it would have been clearer if I had included at least one more of the a variables, or said it like this instead: "...we have ##a_k=0## for all integers k such that ##1\leq k\leq n## ". I thought it was obvious enough that this was what I meant). Maybe an example will make things clearer. Consider the vector space (over ℝ) of polynomials of degree 2 or less. Let f,g be defined by ##f(x)=x## for all ##x\in\mathbb R## and ##g(x)=x^2## for all ##x\in\mathbb R##. The set {f,g} should be linearly independent, right? This is what we have to prove to verify that it is:

    For all ##(a,b)\in\mathbb R^2## such that ##af+bg=0##, we have ##a=b=0##.

    Linear independence of a system of linear equations can be viewed as a special case of the kind of linear independence I defined. Consider e.g. the equations
    a_{11}x_1+a_{12}x_2+a_{13}x_3 &=y_1\\
    a_{21}x_1+a_{22}x_2+a_{23}x_3 &=y_2.
    \end{align} This system is just another way of writing the matrix equation
    $$\begin{pmatrix}a_{11} & a_{12} & a_{13}\\ a_{21} & a_{22} & a_{23}\end{pmatrix}\begin{pmatrix}x_1 \\ x_2 \\ x_3\end{pmatrix} =\begin{pmatrix}y_1 \\ y_2\end{pmatrix}.$$ The set of linear equations is said to be linearly independent if the set
    $$\big\{\begin{pmatrix}a_{11} & a_{12} & a_{13}\end{pmatrix},\begin{pmatrix}a_{21} & a_{22} & a_{23}\end{pmatrix}\big\}$$ (where these 1×3 matrices are viewed as members of the vector space ℝ3) is linearly independent.

    Edit: I won't be making any more posts tonight, so if anyone has questions about this, I hope someone else will answer them.
    Last edited: Dec 2, 2012
  12. Dec 2, 2012 #11


    User Avatar
    Science Advisor
    Gold Member
    2017 Award

    A linear equation defines a hyper plane of Euclidean space.

    A system of linear equations defines a set of hyperplanes. Their solution set is the points in their common intersection.

    If these planes have no common point of intersection then the system is said to be inconsistent.

    If one of the planes does not alter the intersection set determined by the others, then it is said to be dependent.
  13. Dec 2, 2012 #12
    I'll have to admit that I have never heard of this definition before. (I mean: linear dependence of sets rather than vectors). Do you have any reference of a book that does this? I would be very interested in reading about t.
  14. Dec 2, 2012 #13
    Hello Fredrik,

    Firstly I read you definition a little to quickly and I see that your penultimate line is in fact consistent with your definition.

    My comment was motivated by the fact that I am used to an alternative definition scheme, which has certain advantages.

    That is 'linear dependence' is defined.

    The advantage of this is that you can then declare that anything that does not meet this definition as linearly independent.

    You have defined linear independence, but it does not follow that anything that does not meet this definition is linearly dependent.
  15. Dec 2, 2012 #14
    My example of {1,∏} and {∏,∏2} can be established as vectors, if you have a vector space.

    By simply employing them as sets I do no need to do this.

    The definition, comes from the Reference Dictionary of Marthematics by Borowski and Borwein.

    I have found it very subtle in some instances the past.
  16. Dec 2, 2012 #15
    Which vector space??
  17. Dec 2, 2012 #16
    I checked your book and there wasn't a mention of linear dependence of sets. They did give the [itex](1,\pi), ~ (\pi,\pi^2)[/itex] example, but it was clear that they mean this as couples in [itex]\mathbb{R}^2[/itex] and not as sets of two elements.
  18. Dec 2, 2012 #17
    Vectors of the form {a,b} with a, b contained in R.

    (I wish I could find the appropriate symbols more easily)
  19. Dec 2, 2012 #18
    You just mean the couple (a,b) ?? That's very different from {a,b}.

    No book I've ever seen talks about sets {a,b} and {c,d} being linearly dependent...
  20. Dec 2, 2012 #19
    When you check such a book sometimes, as in this case, you need to check more than one definition.

    You will find the mention of sets in the cross-referenced definition of 'linear combination'.
    You have to follow the chain of definitions through.

    However the entry did reference sets in another way as well.

    Now sets have elements do vectors?
  21. Dec 2, 2012 #20
    OK, so here are the two relevant definitions:

    OK, I'm still not seeing anything about sets [itex]\{1,\pi\}[/itex] and [itex]\{\pi,\pi^2\}[/itex] being linearly independent... And I'm also not reading anything about equations being linearly independent.

    No, vectors do not have elements in general.
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook