In Wald's book he write:
"Thus, the distinction between the index notation and the component notation is
much more one of spirit (i.e., how one thinks of the quantities appearing) than of
substance (i.e., the physical form the equations take)."
So,don't worry it much .
We mathematicians tend to prefer to think of tensors in coordinate-free notation. And we make fun of physicists for all the index nonsense, and Einstein summation conventions. So, Penrose, trained as a mathematician, came up with the abstract index notation as a sort of compromise between the index notation and the coordinate-free notation. But, I think it's kind of just formalizing the way physicists were thinking of it all along.
As a mathematician who tries to follow physics, I had to get used to all those indices. It used to be really annoying to see someone write something like
w
i = T
ij V
j
rather than my neat little matrix notation without all those horrendous indices and awful summing over repeated indices:
w = Tv
Nice and coordinate free and no stupid summing (at least not explicitly, until you have to actually compute something)!
But, Penrose's point was that, if you think about it, those two ways of writing the same thing aren't so different. You just stop thinking of w
i as being the components of the vector w and just think of it as the vector w itself. The indices are just markers that act as slots or things you can plug into slots. So, you're thinking of tensors more as functions than as components.
The subscript indices stand for covector slots, and covectors are things that eat vectors. So, subscripts are slots. Superscripts stand for vectors, so they are things that get put into slots. So, essentially, you are getting the best of both worlds because you can think in a coordinate-free way, but the indices are still there to refer to if needed.
Another idea of Penrose was the diagrammatic tensor notation, which is hard to describe here, but I find it helpful, and these sorts of tensor diagram games have, by now, taken a life of their own (in my own field of quantum topology, we actually interpret the diagrams as geometric objects ("colored ribbon graphs" with knots and links as a special case)). I'm tempted to say these tensor diagrams are the same things as Feynman diagrams, but it's really more like a slight variant of Feynman diagrams. Essentially, the idea is just to draw a tensor as a little blob with some arms and legs. The arms are contravariant slots (V*) and the legs are covariant (V). Then, you can glue the diagrams whenever you see repeated indices, so you sum over all the guys that got glued together. That makes the "slot" idea a little more explicit, although, you can also think of it as putting a vector into a function that takes a bunch of vectors as inputs.
For a complete treatment of the abstract index notation, refer to Penrose and Rindler's Spinors and Space-time volume 1. But maybe that would be overkill.