Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

What good is linear algebra?

  1. Mar 26, 2007 #1
    So I'm about 10 weeks into my first course on linear algebra. Things are going well I sopose, I'm finding it much easier than diff eq or calc, but I'm also finding it to be terribly boring.

    Unlike many other math courses, this linear algabra course (and text) has yet to present any applications whatsoever. Everything is just plain math problems. This is quite a change from diff eq and calc, where nearly every concept is presented along with at least one, if not several possible applications (even if many of them are merely trivial).

    Now sure, I can think of lots of applications where being able to solve a large number of linear equations is useful, for example circuit problems, or analyzing truss members, or even differential equations with constant coefficients. But for the simple purpose of solving a large system of linear equations, matlab does the trick nicely, and it certianly does not require a semester of instruction to learn how. In fact, I knew how to solve systems of linear equations using my TI-83 long before I even knew what linear algebra was.

    From my (limited) experience, it seems that the truley interesting and meaningful things in this world are rarely linear. And for those few things that are linear, matlab, or similar programs will take care of them in short order.

    I bet by now you're thinking "isn't this guy ignorant, I can think of hundreds of things that linear algebra is good for." Well I hope that is what you're thinking. And I hope you'll take a couple minutes to explain to me a few of these things.

  2. jcsd
  3. Mar 27, 2007 #2


    User Avatar
    Homework Helper
    Education Advisor
    Gold Member

    How about curve fitting? You would in real-life practice use some software program to accomplish it. If you were the developer who created the software then you would need to understand linear algebra.

    How about some equations you might use in some real life situation which might take the form of linear equations, and you set-up more than one equation and that they be linear equations? Take your pick of the applicable situation - maybe a mixing/concentration problem situation. They DO occur.

    I am not any sort of specialist with linear algebra. I merely know that they can be used in realistic situations. Knowing about this kind of algebra can be a good thing. also, don't always expect the system to be complicated. (want to know more? write me PM)
  4. Mar 27, 2007 #3


    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    Basically, we are very good at doing linear algebra over a field. So much that it is often worthwhile to spend a great deal of effort transforming a problem into a linear problem.

    For example, that's exactly what differential calculus is. :wink:

    Being good at linear algebra goes beyond simply solving equations -- concepts like rank, nullity, diagonalization, trace, determinant, and so forth are very powerul tools for understanding and describing things.
    Last edited: Mar 27, 2007
  5. Mar 27, 2007 #4


    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

    Some assorted comments:

    Numerical errors can fool you into thinking that the determinant of a certain matrix is nonzero when it is really a singular matrix.

    While matlab might be good for numerical computation of matrices, one needs linear algebra to formulate problems that may be given to matlab. If your system of equations has some special properties, you may be able to use linear algebra to simplify a problem or optimize a computation that the generic matlab routine would not take.

    One needs some familiarity with linear algebra to properly formulate standard quantum mechanics or analyze special relativity. A numerical tool may help visualize certain features... but without the foundation of linear algebra, one basically has a bunch of numbers or graphs.. and no closed form expressions which may be easier to analyze, classify, and interpret.
  6. Mar 27, 2007 #5
  7. Mar 27, 2007 #6

    matt grime

    User Avatar
    Science Advisor
    Homework Helper

    The first thing to do is to stop you thinking that the reason you learn linear algebra is to solve simultaneous equations - a trained monkey can do that. It is to teach you why you an solve some systems and not others, it is to study vector spaces, and more importantly maps between vector spaces.

    By analogy - anyone can soolce x^2=2 by plugging it into their calculator and not bothering to think *why* there is a solution at all.
  8. Mar 27, 2007 #7


    User Avatar
    Science Advisor

    Indeed, any thing that involves "linearity" (linear differential equations, Lie Algebras, linear integral equations, etc.), the simplest of all kinds of problems, has all of Linear Algebra as its basic theory.

    Of course, I was tempted to respond that if you idea of "real life" is saying "Do you want fries with that", then Linear Algebra is of no use at all!
  9. Mar 27, 2007 #8
    There are lots of applications in physics. I will give two modern ones:

    i, we often do quantum mechanics in the languages of vector spaces and linear operators; and
    ii, linear algebra is the basis of multilinear algebra which is the basis of what a tensor is and these concepts are extremely important in general relativity.
  10. Mar 27, 2007 #9


    User Avatar
    Science Advisor
    Homework Helper

    repeating hurkyl's and masudrs remarks, we are only really good at solving linear problems. since as you have observed, most interesting problems are non linear, the primary techniqur for attacking them is to approximate them by linear problems. this is called differential calculus.

    a manifold, e.g. is a typical non linear geometric construct used in physics. the tangent bundle, cotangent bundle, and various higher tensor bundles, are different spects of the process of linearizing the study of a manifold.

    tensors go up in complexity one step at a time, linear approximations, bilinear approximations, trilinear ones, etc, but even the study of say bilinear tensors uses the fact they themselves form a linear vector space. the duality between vectors and covectors is obtained by regarding one as linear functions on the other.

    linearity is the most useful and pervasive concept in all of mathematics. in abstract algebra, abelian groups are classified by looking at them as a sort of linear space whose coefficients are integers, then representing the group as a matrix, and diagonalizing the matrix.

    this same technique then applies more generally to other abelian groups, such as vector spaces, which are acted on by a ring such as a polynomial ring. this is done by fixing a single linear transformation to act as the variable X.

    In this way one can use the techniques of linear algebra to study itself, diagonalizing a matrix of polynomials to analyze a matrix of scalars.

    in diff eq, the fundamental tool for studying non linear systems is to inearize them. indeed the same formula for the solution of a linear system, in terms of exponentiating matrices, can be used to give a solution to a non linear systems as well, as shown beautifully in arnol'ds book on ordinary diff eq.

    here is a simple way linearity simplifies non linear objects: the chain rule properly stated says the derivative of a composition of non linear functions is the composition of their linear derivatives. and on and on.....

    by the way the one thing i seldom do with linearity is solve specific systems of numerical linear equations. but sometimes i need to know a determinant. as artin says, the usual way to deal with linear algebra is to throw the matrices out, unles a specific calculation is needed.
    Last edited: Mar 27, 2007
  11. Mar 27, 2007 #10


    User Avatar
    Science Advisor
    Homework Helper

    another tiny thing: the concept of linear independence, bases, and linear dimensions is a way to represent infinite spaces, namely finite dimensional vector spaces like R^3, in terms of a finite amount of data.

    i.e. even if the space has an infinite number of vectors, they can all be represented as linear combinations of a finite subset.

    the theory of cohomology of sheaves is sometimes called global linear algebra, since it also helps linearize the study of analysis and geometry.

    e.g. on complex space C^n, the exponential map takes holomorphic complex valued functions, elements of the sheaf O, to never zero ones, elements of O*, and the kernel of exponentiation is (up to a multiple of 2pi), the sheaf of integer valued functions Z.

    the exact sequence 0-->Z-->O-->O*-->1, yields the long exact sequence of cohomology 0-->Z-->C--->C*-->H^1(Z)-->H^1(O)-->H^1(O*)-->H^2(Z)

    and since for complex affine space C^n, the topological groups H^1(Z), H^2(Z) are zero, this equates the vanishing of the analytical groups H^1(O) and H^1(O*). Then the solution of the dbar diff eq, i.e. the "cousin problem", gives the vanishing of H^1(O), and hence the vanishing of H^1(O*).

    But this last group measures the non triviality of line bundles, i.e. divisors, on C^n. Consequently, every divisor on C^n is the divisor of a meromorphic function, which solves the higher dimensional mittag leffler problem.

    the linear algebra is in the exact sequences, reducing the analysis down to its essential, solving the cousin problem, or dbar differential equation.

    so in linear algebra what is important is the concept of linearity, the rank nullity principle, eigenvectors, diagonalization, jordan form, rational normal form, changing bases, behavior of determinants, i.e. properties of linear maps, not reducing specific linear systems.
    Last edited: Mar 27, 2007
  12. Mar 27, 2007 #11
    I'm curious about what you mean by this, could you elaborate? What does "represent the group as a matrix" mean here?
  13. Mar 28, 2007 #12
    I haven't yet studied this, so I don't know the specifics, but I figured that I'd tell you what I know so that he can elaborate further.

    One result from Linear Algebra is that any linear map can be represented as a Matrix. Because of this, any finite group can have its elements represented as matrices.

    For instance, one representation of the group {0,1} is
    [tex]\left[ \begin{array}{c c} 1 & 0 \\ 0 & 1 \end{array}\right],\left[ \begin{array}{c c} 1 & 0 \\ 0 & -1 \end{array}\right][/tex]
    Note that just like these matrices, 0+0=0, 0+1=1, 1+1=0 (However, you use matrix multiplication instead of addition)
    Last edited: Mar 28, 2007
  14. Mar 28, 2007 #13


    User Avatar
    Science Advisor
    Homework Helper

    A matrix representation of a group G is a homomorphism from G into GLn(F) (where F is some field). mathwonk was basically hinting towards representation theory.
  15. Mar 28, 2007 #14
    ah, that makes sense. Though the English word "representation" makes me think it should be an isomorphism rather than just a homomorphism.
  16. Mar 30, 2007 #15
    Thanks for all the replies. I'm starting to see there might be more to linear algebra than just solving equations. The articles about google are quite interesting. Definently a legitimate application, yet as concrete as the basic concept seems to be it's still quite difficult to concieive on such a large scale.

    So may I pose another question?

    We've recently spent most of out time learning about vector spaces and subspaces and their relations. Some of these things I can see, others I can't. I can think about a space or subspace, and as long if it's only in R3 I can see a line, a plane, or the whole space. I can also think about two vectors that make a plane in R3, and I can see how the vector b may or may not be a combination of those two vectors.

    I'm ok with the column space, I can see that most of the time. I think about the row space, and I (think) I can see that too, but I don't see it in relation to the column space, it see it as a seperatre entity. I can see the null space and left null space, but only in the sense that I see them as the column space of their basis. In my mental vision I fail connect these. I see them, but I see them seperately. Similary, I can see A, and I can see the inverse of A, or the transpose of A, but I see them all seperately.

    In trying to make these types of connections with other concepts I often imagine physical interpretations (I've gotten miles of use out of the whole block-spring system). But with linear algebra I have found few. So I asking for some type of physical interpretations. What does the null space mean in the physical realm? I understand that (in math terms) the left null space is just the null space of the transpose of A, but what is the physical meaning of this so called transpose? What about rank? Or Inverse?

    Ok, so maybe that's more then just a question, it's a few questions. Once again your replies are appreicated. And just for the record my major is mechanical, hence the "I like to see things move" mentality.
  17. Mar 30, 2007 #16

    matt grime

    User Avatar
    Science Advisor
    Homework Helper

    A matrix is just a \emph{linear map}. A map between vector spaces that preserves the nice properties ofa vector space. Normally, I would use column vectors for the vector space and multiply on the left. So it would be written like


    with A sending v in V to Av in W, for V,W two vector spaces and A linear map.

    I'm glossing over something important about bases, but we can forget that for now. But I just want you to be aware of it in case someone mentions it later.

    The kernel of A is the subspace of V that A sends to 0. This can be calculated as a null space of the matrix.

    Some people like to use row vectors, and multiply matrices on the right. Thus v would go to vA, and you'd have to use the other null space (I don't tihnk in these terms and can't recall which way round you want to write that).

    There is an obvious way to get between the two ways of thinking: taking transposes.

    There is no reason why the left null space and the right null space of A should be at all related, by the way. As it turns out there are some relations on the dimensions of things, but that is all.

    For inverses, we have to have a square matrix - a linear map from V to V. The matrix is the inverese linear map. So if A is a rotaion about the z axis by 90 degrees clockwise, then the inverse is rotation 90 degrees anticlockwise about the z axis (or counterclockwise if you prefer US english).

    If you want even more physical meaning for transpose, then you should look up things about inner-products, or wait for them to come into the course.

    If you limit yourself to only things you can visualize you'll not get very far. In particular quantum mechanics will forever be a closed book since it is about things in linear algebra that are inherently unvisualizable.
  18. Mar 31, 2007 #17
    Stevecgz, I asked myself similar questions when I was in my first and second year of maths and prefered the applied side of maths. Now, a few years down the line and doing high energy physics I wish I'd paid more attention back then. Linear algebra underpins so much of quantum mechanics and relativity. There's bearly a part of physics you can't use linear algebra in (I struggle to think of one). Apart from the rigorous proof courses like analysis (and even that's good for having a well rounded education), pure maths is not as pure at it might appear to an undergrad. Just this week I've read two papers on algebriac geometry's application to string theory, providing managable ways to attack previously complex, to the point of impossible, problems. Now I wish I'd not skipped all those Friday afternoon 'Groups, Rings and Modules' lectures in my 2nd year!

    The things of linear algebra like kernals and endomorphisms etc are not so much literally physical things but more an expression of an underlying structure to the system. A smooth linear map of a space onto itself isn't literally a thing but perhaps an expression of how one configuration flows smoothly into a different configuration. Some regions will shrink to a point at the origin (the kernel), others will remain fixed (eigenvectors), that sort of thing. Better to physically picture things like that than saying "What's a matrix look like in reality" which isn't going to get an answer.
  19. Apr 1, 2007 #18


    User Avatar
    Science Advisor
    Homework Helper

    if G is a fin itely generated aselian group, there is a surjective homomorphism Z^n-->G, for n = the number of generators. then if K is the kernel of theis homomorphism, the fundamental homomorrphism theorem says that G is siomorphic to Z^n/K.

    But K is also a finitely generated abelian group, in fact with at most n generators also, so there is another homomorphism Z^n-->Z^n whose image is K. If M is the matrix of this last homomorphism, then G is isomorphic to the cokernel of M.

    thus we have represented G as isomorphic to the cokernel of a square matrix of integers. Simplifying, i.e. diagonmal;ioziong thism matrix lets us write G as a direct product of cyclic groups.

    If T is any linear transformation of a finite dimensional vcector space V over a field k, we get a k[X] module structure on V by letting multiplication by X mean action by T.

    Then we can reprersent this module structure via a square matrix of entries in k[X]. diagonalizing this matrix lets us find the jordan form, or rational canonical form of T.
  20. Apr 21, 2008 #19
    My first post in this forum :)
    So far, the most common application I have found for linear algebra (with the help of differential equations) are the mixing problems. I'm not very good at theory so I'll just copy an interesting example (directly from a book):

    Two tanks(1 & 2) each containing 20 L of a solution consisting of salt dissolved in water. A solution containing 4 g/L of salt flow into a tank 1 at rate 3 L/min and the solution in tank 2 flows out at the same rate. In addition, solution flows into tank 1 from tank 2 at a rate 1 L/min and into tank 2 from tank 1 at rate of 4 L/min. Initially tank 1 contained 40g of salt and tank 2 contained 20 g of salt. Find the amount of salt in each tank at time t.
    (Sorry if I don't copy the solution... it's just too much)

    Wouldn't it be nice to be able to solve a real problem like this just by using those systems of linear equations and the calculus that you already know? BTW, if you are seeking a major as a chemical or industrial engineer you will want to know that.
  21. Apr 21, 2008 #20


    User Avatar
    Science Advisor

    It's not just "mixing problems". The whole theory of "linear differential equations" is based on "linear algebra". That's why Linear Algebra should always be a prerequisite for Differential Equations.

    In fact, any thing involving "linearity"- the basic idea of which is that we can separate a problem into parts, solve the parts separately and then put the solutions together to solve the entire problem- requires linear algebra. It is used in differential equations, computer graphics (moving the viewpoint in 3 dimensions), quantum mechanics (the Dirac Formalism), economics (linear programming), etc.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook