Diagrammatic Tensor Notation from the Beginning

26 replies · 12K views · last reply
LukeD
Messages
354
Reaction score
3
I really liked Penrose's diagrammatic way of writing Tensor algebra, so I spent a while learning the basic notation. Unfortunately, it took a very long time for me to learn this because there is so little info on it to begin with. I also didn't see much mention of how to use the notation for doing numerical computations.

So since I want to be able to use the notation for everything that I should be able to do with tensors, I figured that I'd start at the very beginning of algebra and use the notation. My plan is to write a little tutorial that teaches everything from how to write numbers, vectors & covectors to how to take integrals.

The notation for adding tensors is mine as far as I know, and I've changed some other notation here and there.

I have 14 pages done, but they are very rough. So far they are very basic ideas. Should I continue with this? Any suggestions for cleaning it up? I want to make it as short & simple as possible while still getting the point across.
 
Attachments
  • Handwritten notes on lined paper titled Numbers: numbers in bubbles, adjacent bubbles added, bubbles joined by a line multiplied, with diagrams and a bracketed note that order does not matter.
    01-Numbers-1-Intro.jpg
    21.5 KB · Views: 1,555
  • Handwritten lined-paper notes on grouping and FOILing using circled numbers in boxes, showing 5 times 6 rewritten as 5 times (2+2+2) and evaluated to 30, and the rule (a+b)(x+y) = ax+ay+bx+by.
    02-Numbers-2-Grouping.jpg
    23.6 KB · Views: 1,409
  • Handwritten notes on lined paper titled Vectors: a vector is drawn as a box with a tag on the end, and examples show a grocery list and ordered pairs of numbers as boxed diagrams.
    03-Vectors-1.jpg
    29 KB · Views: 1,312
Last edited:
  • Like
Likes   Reactions: ionsme

Discussion

Pages 4-6
 
Attachments
  • 06-Functions-2.jpg
    06-Functions-2.jpg
    30.3 KB · Views: 1,001
  • Handwritten notes titled Functions of Vectors, with boxed diagrams: price box containing groceries box, equals 20; a box with a semicircle for tags; a 2 attached to price, equals 40; and a total price box equal to retail price box plus tax box.
    05-Functions-1.jpg
    30.4 KB · Views: 1,050
  • Handwritten notes titled Rules of Vectors with diagrams: vectors are identified by tags, vectors can only be added to other vectors, grouping vectors with a tag, and vectors distributed over numbers using boxes with apple and pear tags.
    04-Vectors-2-Rules.jpg
    22.1 KB · Views: 1,137
Pages 7-9
 
Attachments
  • Handwritten notes titled Covectors: a covector is a special type of vector function, shown as a box labelled 1 for apple with a line-and-cap input, plus diagrams with Groceries; a covector takes 1 vector and outputs a number; vectors can be grouped and distributed over covectors like numbers, with a sack of apples diagram.
    07-Covectors-1.jpg
    32.5 KB · Views: 1,206
  • Handwritten note reading "We can also push numbers between vectors and covectors." Below are two box diagrams: a box labelled Price, a covector symbol and Groceries, with a circled 2 at the right end, above an equals sign and a circled 2 at the left end, Price, the covector symbol and Groceries.
    08-Covectors-2.jpg
    24.3 KB · Views: 929
  • Hand-drawn sketch of a refrigerator titled "Contents of Your Fridge" with a box labelled "Donuts" marked 12, a box labelled "Donut" marked 1, and a bottle labelled "apple" marked 5, each with a numbered oval tag.
    09-Vector Example-1.jpg
    25.1 KB · Views: 983
Pages 10-12
 
Attachments
  • Handwritten notes on lined paper: a boxed expression '(# apple(s))^2 + 3' next to a boxed 'a ⊗ covectors' with a circled label, captioned 'Not a covector'. Below, 'Don't expect anything crazy to be a covector. Covectors are simple.' with 'simple' in green.
    12-Not Covectors-2.jpg
    17.3 KB · Views: 890
  • Handwritten notes on lined paper headed 'Beware! Not every function that sends a vector to a number is a covector!' with two examples: a price box and a function box with a vector input, and a second pair of boxes for buying an apple and getting a t-shirt free, each showing function inputs and outputs.
    11-Not Covectors-1.jpg
    24.4 KB · Views: 938
  • Hand-drawn diagram on lined paper: a large box labeled Price containing a circle with 2 connected to a box reading $ donut, a circle with 1 connected to a box reading $ for apple, and a nested box labeled Buying containing a circle with 300 connected to a box reading $ hack saw, with arrow-like connectors.
    10-Covector Example-1.jpg
    32.6 KB · Views: 952
Pages 13 & 14

I'm thinking of talking about bases next & then tensor products.
 
Attachments
  • Handwritten notes titled 'Zero' explaining the zero covector: a box labelled 0 with a zero covector label, a vector x fed in giving 0, and 2 for apple plus -2 for apple equalling 0 for apple.
    13-Zero Covector.jpg
    23 KB · Views: 938
  • Handwritten notes on ruled paper explaining the zero vector: a box labelled O with a hook, a box X hooked to box O giving a circle O, and a boxed sum of Y and minus Y equalling a box labelled O.
    14-Zero Vector.jpg
    24.4 KB · Views: 908
It looks pretty good. I've always imagined these objects fitting together sort of like toys. I wouldn't try to actually use the diagrammatic notation to do any real calculations, though...but as a teaching tool, it might be nice. Like teaching linear algebra to elementary school kids...
 
6 More pages. I decided to do tensor products and contraction before talking about basis vectors and doing numerical calculations.
 
Attachments
  • Handwritten cartoon on lined paper titled 'Please Label': a box with T and labels e, i1, i2, o, showing a tensor with labelled inputs and outputs. Caption reads 'A mighty tensor with labels!' Speech bubble: 'Uh oh, I hope these all do the same thing.' Caption below: 'Otherwise, we can get very confused.'
    17-Please Label.jpg
    27 KB · Views: 912
  • Handwritten notes on lined paper showing tensor network diagrams: boxes labelled X, Y and Z with legs, first equated to Z, X, Y reordered, then a comparison of two wirings of X, Y and Z marked as not equal, and a box with Z, X, Y grouped together.
    16-Connect with Care.jpg
    27.8 KB · Views: 926
  • Handwritten notes titled Tensor Product explaining that two covectors can be tied together into a tensor product, with a box diagram, then a covector connected to a partner multiplying the rest, and the word TENSORS at the bottom.
    15-Tensor Product.jpg
    24.7 KB · Views: 1,036
pages 18-20
 
Attachments
  • Handwritten notes titled ADDING TENSORS in green and blue, showing two boxed tensor diagrams with input and output legs, and a crossed-out box with mismatched legs and a circled 3 labelled none of those things add.
    18-Adding Tensors.jpg
    29.9 KB · Views: 963
  • Handwritten note on lined paper reading 'Tensors add the same way as everything else.' Below are two pairs of diagrams: a box holding an X and a Y tensor with index slots A, B, C, joined to a rounded piece with indices 1 and 2, with an equals sign between them; and a second box with X and Y tensors joined to pieces labelled 1, 2 and a free index B.
    19-Adding Tensors-2.jpg
    24 KB · Views: 910
  • Handwritten notes on lined paper titled Contraction, with a box diagram of tensor X with one input and one output joined by a loop, and a second box diagram showing two tensors X and Y connected in a sum-based contraction equated to a combined diagram.
    20-Contraction.jpg
    23.7 KB · Views: 867
12 More.
I got down everything that I wanted for basis vectors, tensor components, and computations.

Here's 21-23
 
Attachments
  • Handwritten lecture notes on linear algebra: a basis is a set of linearly independent generators, shown as three boxed vectors labeled as a basis for 3D vectors; two bases have the same number of vectors, n and m, with n = m, called the dimension.
    23-Basis.jpg
    20.9 KB · Views: 842
  • Handwritten notes on lined paper defining linear dependence: a set of vectors {v1, v2, ..., vn} is linearly dependent if there are numbers a1, a2, ..., an not all zero that combine the vectors to give the zero vector, and linearly independent if the only such combination uses all ai = 0, with boxed diagrams of the sums.
    22-Linear Dependence.jpg
    19.9 KB · Views: 834
  • Handwritten notes on lined paper: a generating set of produce vectors apple, orange, pear; a generating set for 3D vectors i, j, k; and groceries = x apple + y orange + z pear.
    21-Generators.jpg
    19.9 KB · Views: 893
Pages 24-26
 
Attachments
  • Handwritten lecture notes on lined paper titled "Our Standard Basis", showing basis vectors e1, e2, e3 and a vector v written as components (2, 1, 0) and as the sum v^i e_i.
    24-Our Standard Basis-Components.jpg
    22.2 KB · Views: 704
  • Handwritten notes on lined paper: 'We can use them to characterize covectors' with three boxed diagrams where w points up to e1, e2, e3 giving 2, 3, 0; below, 'And then compute numbers' with v = column of 2e1, 1e2, 3e3 and v to w = 2e1 dot w, 1e2 dot w, leading to 2*2 + 3 = 7.
    25-Characterizing Covectors.jpg
    19 KB · Views: 831
  • Handwritten notes on lined paper defining a dual basis of covectors: three boxed covectors each paired with a basis vector, giving 1 when indices match and 0 otherwise, written as the Kronecker delta epsilon-i applied to e-j equals delta-ij.
    26-The Dual Basis.jpg
    19.1 KB · Views: 797
Pages 27-28

We dissect the identity
 
Attachments
  • Handwritten notes on lined paper with a cartoon story: 'With a basis + a dual basis, you can rule the world (of tensors)'; a dual basis described as a gang that breaks tensors into parts, a basis described as well-natured folk to build a kingdom from, and tensors as strange beasts to tame or destroy, shown as a toothy monster.
    27-For Fun-The Cast.jpg
    19.1 KB · Views: 861
  • Handwritten notes titled How to Tame a Tensor, with a stick figure next to the text about the identity: a line with a loop in the box-and-wire tensor diagram notation, followed by equations showing box T with wires equal to T with extension cord, and a circle attached to itself.
    28-Taming the Identity-1.jpg
    18.2 KB · Views: 925
  • Handwritten notes with a stick-figure character and a cartoon creature; the text reads 'Fortunately, I've got some basis and dual basis vectors in my Mathemagician's Tool. We can direct the little buggers!' followed by a diagram of three rows of wire-and-box connections equal to the same wires, and a final sketch of a single wire fork.
    29-Taming the Identity-2.jpg
    19.5 KB · Views: 859
Pages 30-32

These are probably the last ones I'll do for a while. The next topic that I want to do is differentiation, but my Differential Geometry is very weak and I want to be more comfortable with it before I go making up notation for it.

Penrose and Cvitanović both have their own notations for derivatives, but I can't figure out Cvitanović's.
 
Attachments
  • Handwritten diagrammatic derivation of a tensor expanded as a sum over indices i and j, with boxes for basis vectors e_i and dual basis epsilon^j and a box T, ending with a_i^j defined as the contraction of epsilon^i, T and e_j labelled Matrix Values.
    32-Disecting a Tensor-2-The Matrix.jpg
    16 KB · Views: 900
  • Handwritten notes titled "Dissecting a tensor": a box labelled T with one top leg, two left legs and one right leg, equals a larger boxed row of three T boxes, each with a labelled small box attached, joined by arrows and dots, with labels e1, e2, en and eps, eps2, epsn.
    31-Disecting a Tensor-1.jpg
    20.4 KB · Views: 920
  • Handwritten page of tensor diagram notation: a bent-arrow node equals a box containing three stacked blocks each with epsilon and e labels, then the text 'But now we know what the little bugger looks like inside. And we can compute its trace!' followed by a trace diagram reducing to 3, with a note about n-dimensional space.
    30-Taming the Identity-3.jpg
    18.6 KB · Views: 863
i certainly appreciate it thoroughly - i looked for a long time for any info on this stuff and couldn't find anything. the closest thing i could was this book:

http://press.princeton.edu/titles/8839.html

which apparently does stuff with the notation.

have you been in contact with penrose?
 
No, I haven't been in contact with Penrose at all.
Cvitanovic's book looks fun, but I didn't get much out of it. He draws lots of fun pictures, but he doesn't explain his ideas thoroughly enough for me to understand much.
 
Excellent! I like the uncluttered style and the everyday examples: shopping lists, special offers, etc. Nice, intuitive one-line definition: "Vectors are objects that can be added together or multiplied by numbers." I'm sure anyone who's struggled to learn what a tensor is from the eponymous Wikipedia entry will find this a welcome relief. Are you going to have appendices giving straighter definitions in more formal language and relating it all to conventional notation? I bought Road to Reality recently, partly out of curiosity about this notation, although I've been diverted down lots of detours, trying to figure out other basic stuff... My favorite captions are "a mighty tensor with labels" and "with a basis & a dual basis you can rule the world (of tensors)".

There were maybe one or two places where an extra word of explanation would help. E.g. on the first page, it says "the way the bubbles are connected with lines" doesn't matter, and illustrates this with a 1 connected by one line to a 2 which is connected by one line to a 3 which is connected by two lines to the 1. But 1 is a special number in multiplication; would it mean the same if there were two lines connecting the 2 and 3, or would that mean 1 times 2 times 3 times 2 times 3?

The other thing that I thought might be a bit confusing is the similarity between the notation of a line connecting boxes for the tensor product (as an alternative to the more obvious notation of just sticking the boxes together), and the notation for contraction, but then I noticed you're version is slightly different from Penrose's in that he doesn't use those connector symbols but shows the difference between vectors and covectors by whether their cables point up or down. So his has the advantage of being related to the conventional index placement, but yours has the greater advantage that it can be read upside down, which suits the spirit of coordinate independence!

> in my Mathemagician's Tool.

Tool Box? Tool Kit?
 
I'm thrilled that you liked it!

Rasalhague said:
Are you going to have appendices giving straighter definitions in more formal language and relating it all to conventional notation?
I probably won't have straighter definitions in terms of vector spaces and the such. You can get those anywhere. I started writing this for my aunt who is dyslexic in hopes that she'll be able to read this notation easier than the usual way of writing algebra. (I've sent her the first pages. She hasn't had time to read them yet.) I'm not writing it for her anymore, but I want to keep things on the level of someone who's had a high school education and probably isn't interested in knowing exactly how things work. However, I expect that most people who read it will already know how everything works anyway.

I might have a few pages on conventional notation if people are interested in my writing more of these.

There were maybe one or two places where an extra word of explanation would help. E.g. on the first page, it says "the way the bubbles are connected with lines" doesn't matter, and illustrates this with a 1 connected by one line to a 2 which is connected by one line to a 3 which is connected by two lines to the 1. But 1 is a special number in multiplication; would it mean the same if there were two lines connecting the 2 and 3, or would that mean 1 times 2 times 3 times 2 times 3?

That's a good point. If I remake that page, I'll be sure to change it to a better example. That was the first page I made, and I wasn't sure what I was going to do about the tensor product notation yet! You'll notice there's a big error. I say "This property is special of numbers", but as long as there are no markers on the lines, it's true of any tensor.

The other thing that I thought might be a bit confusing is the similarity between the notation of a line connecting boxes for the tensor product (as an alternative to the more obvious notation of just sticking the boxes together), and the notation for contraction, but then I noticed you're version is slightly different from Penrose's in that he doesn't use those connector symbols but shows the difference between vectors and covectors by whether their cables point up or down. So his has the advantage of being related to the conventional index placement, but yours has the greater advantage that it can be read upside down, which suits the spirit of coordinate independence!
In the usual notation, you tensor everything together before you do contractions anyway. So it doesn't make a difference. And in my notation - if it's connected by a line in some way, it gets multiplied.

> in my Mathemagician's Tool.

Tool Box? Tool Kit?
It's like a Leatherman Tool http://www.protoolreviews.com/reviews/hand-tools/wrenches-sockets/leatherman-super-tool-300-multitool/image_thumb
but with Math Magic

And what about "Our Standard Basis"?
Maybe I should have called it "This Standard Basis"
 
The monster tensor that needs to be broken down and destroyed (great drawing, by the way) reminds me of the apology that Kip Thorne makes in one of his online lectures about all the bloodthirsty imagery; he'd just been talking gleefully about "strangling" tensors.

LukeD said:
In the usual notation, you tensor everything together before you do contractions anyway. So it doesn't make a difference. And in my notation - if it's connected by a line in some way, it gets multiplied.

Yeah, it's not ambiguous. I just thought it might be easier to take in a diagram at a glance if the only lines connecting them were contractions, and the tensor product was shown only by the other method you use: sticking the boxes directly together.

LukeD said:
It's like a Leatherman Tool http://www.protoolreviews.com/reviews/hand-tools/wrenches-sockets/leatherman-super-tool-300-multitool/image_thumb
but with Math Magic

Ah, I see. Your Swiss Army tensor-filleter.

LukeD said:
And what about "Our Standard Basis"?
Maybe I should have called it "This Standard Basis"

If the underlying set of your manifold is Rn, then I suppose the identity transformation offers a natural choice of coordinate system, the standard basis. Is this the only case where it makes sense to talk about the standard basis? If you have a manifold over n-dimensional Euclidean space conceived of as an affine space, then there's no natural origin and no natural orientation; each of the inifinite number of possible orthonormal bases has a sort of naturalness about is, as the simplest sort of coordinate system, but no one of them is more natural or standard than another. And similarly I think for the generalisation to Riemannian manifolds, but I'm relatively new to all this, and not any kind of expert...

(It gets a bit confusing because some people use the name Euclidean space for the inner product space made of Rn with the usual rules, whereas others reserve the name Euclidean space for an affine space whose associated vector space is made up of Rn with the usual rules.)
 
Hm... I'm still learning Differential Geometry, but I'm trying to incorporate the various differentiation operators.

One thing I've done is taken the advice of getting rid of using lines for tensor product. Now lines only indicate contractions or free indices.

One thing that I've noticed is that I can interpret the circle on the end of a vector as being a slot to insert a real valued function on a manifold. If I make that identification, then the identity operator works perfectly as the gradient of a function. Inserting a function into the slot on the identity assigns the correct covector.
 
I must say that I read only the first page, and I found it quite hilarious.
associating numbers with bubbles... :-)
 
I'm trying to incorporate the ideas from Differential Geometry into my notation. For real valued functions (exterior differential calculus), my notation seems to work pretty well, I'm able to do all of exterior algebra without any pesky (-1)^a terms appearing in the diagrams for the wedge product or exterior derivative at all. (There's no such thing as a "graded Leibniz rule" in the diagrammatic notation, just the normal Leibniz rule.)

My notation seems a little unwieldy though in the non-commutative geometry case of the differential geometry of Lie algebra valued functions since I have vectors in 2 different spaces to worry about. I'm also not too satisfied with my notation for the Lie bracket... It shows up constantly in non-commutative geometry, so it would be nice to have a good notation for it.

I'm currently writing up some very rough pages to show what I mean. I'll have them up shortly. Hopefully even though they're very rough, someone will be able to understand them and give me some suggestions.
 
It should be noted that I'm using a circle to represent values of one space and a diamond to represent values of a second space.
 
Attachments
  • Handwritten notes titled Differential Geometry: vectors are differential operators on the space of functions on a manifold; f(p) is a function on a manifold returning a real number or a vector in another space W; vectors obey the Leibniz rule shown with boxed v and circled f(p), g(p) diagrams.
    01-Differential Geometry.jpg
    19.7 KB · Views: 708
  • Handwritten tensor diagram notes: a box labelled T with upper index i and lower indices, equated to a sum over i of boxes labelled dx^i and dx_i; a second diagram with box L and indices i, j, k, with dx and partial x labels; text reads 'In general, the component derivative may depend on the bases used! But in some cases it doesn't.'
    02-Component Derivative.jpg
    14.5 KB · Views: 679
  • Handwritten notes titled "Lie Bracket" with box-and-circle diagrams of vector fields acting on functions, a note that the result is not a vector in general, and the Lie bracket written as [v, w] with its derivation form.
    03-Lie Bracket.jpg
    20.6 KB · Views: 645
I don't know if I like my notation for the Lie bracket. I haven't looked at the Jacobi identity yet though. I'd like to find a notation that makes the anti-symmetry of the Lie bracket and the Jacobi identity easy to express.
 
Attachments
  • Handwritten notes on graded Leibniz rule using index diagrams: a normal Leibniz rule with alpha and beta boxes, then the graded version with a sign (-1) raised to the power of |alpha|, and a note that grading appears when indices are not placed consistently.
    06-Grading and Diagrams.jpg
    15 KB · Views: 703
  • Handwritten page of hand-drawn diagrams for the Lie bracket: two boxed-and-circled diagrams labelled Lie Bracket in V and Lie Bracket in W, then a question 'What is good notation for this?' above candidate diagram notations with question marks and equals signs.
    05-Lie bracket notation.jpg
    12.5 KB · Views: 704
  • Handwritten notes on the Lie bracket with diagrammatic tensor boxes: a box T with vector-valued inputs and output, the tensor product replaced by a vector product in W, and a diagram of the Lie bracket of two vectors in W.
    04-Lie valued forms.jpg
    14.7 KB · Views: 612
Ok, here's the last one for now. Is there anyone who understands this stuff better than I who could suggest some better notation for vector valued objects and for the Lie bracket?
 
Attachments
  • Handwritten string diagrams with stick-figure nodes for a Lie bracket of forms a and b, with the text 'No grading!' and 'Usual way of writing this has grading'.
    07-Grading and Lie Valued Forms.jpg
    16.1 KB · Views: 739
Making the vector type symbol smaller makes the Lie bracket much cleaner, but it looks a lot like a contraction between a vector and covector. That could get confusing but anything I've tried to do to clear up that part just looks too cluttered. :(
 
Attachments
  • Handwritten page of diagrammatic notation for Lie brackets: a boxed bracket of boxes A and B equals a circled A and B diagram, followed by a line of antisymmetry, a vector-valued functions section, and the Jacobi identity drawn as three circled-node diagrams equal to zero.
    08-Another Lie bracket notation.jpg
    12.1 KB · Views: 667
I believe that I have just written the Jacobi Identity in the coolest way that will ever be discovered.

Seriously, this picture is so beautiful that it should be engraved somewhere.

This picture is supposed to be a 3D shape by the way, but i can't draw it too well. If i could, I'd have all the As together, all the Bs together, and all the Cs together, and then the addition point up at the very top hovering over the whole shape.
 
Attachments
  • Hand-drawn tensor diagram with three labelled strands A, B and C joined at vertices, a square node with small circles, and an equals sign leading to a box containing a circle with a short stem.
    Jacobi Identity.jpg
    21.4 KB · Views: 686
  • Like
Likes   Reactions: ionsme and JonnyMaddox

Similar threads

Undergrad Is Diagrammatic Tensor Notation Widely Used in Mathematics?
  • Hill
  • · Replies 33 ·
  • Differential Geometry
2
Replies
33
Graduate Penrose diagrammatic tensor notation tutorial from basics
  • LukeD
  • · Replies 1 ·
  • Special and General Relativity
Replies
1
Graduate Trying to understand spin networks, or at least SU(2)
  • Coin
  • · Replies 14 ·
  • Beyond the Standard Models
Replies
14