What about the case when you circumnavigate the Earth around the Equator. Would not the arrow return to the starting point pointing in the same direction it started in?
Here's the simplest case I can think of to show how parallel transport works. As you say, consider circumnavigating the Earth. And you're right that if you go around the Equator (or any geodesic) that a parallel-transported vector will come back to its original value, but now consider a non-geodesic path, along a line of constant latitude θ. As you know, a pilot following such a curve will have to keep his rudder turned constantly to one side, to stay on course. And I say that in this case a parallel-transported vector will not come back.
The metric for a sphere is ds
2 = R
2(dθ
2 + sin
2θ dφ
2). Let v be the unit velocity vector along the circle. It will have components (0, (R sinθ)
-1). The parallel transport equation for any vector w is w
μ;νv
ν = 0, or w
μ,ν + Γ
μνσw
νv
σ = 0. The only nonzero Christoffel symbols are Γ
θφφ = - sinθ cosθ and Γ
φθφ = + sinθ cosθ. If we take w
μ = (a, b), the equations for a(φ) and b(φ) are da/dφ - sinθ cosθ b = 0, db/dφ + sinθ cosθ a = 0 with solutions a(φ) = A sin Ωφ, b(φ) = A cos Ωφ, where Ω = sinθ cosθ.
What does this show? It shows that a parallel-transported vector will precess with constant angular velocity Ω = sinθ cosθ, where θ is the latitude. By the time we have gone completely around the Earth, φ has increased by 2π and w has precessed by 2πΩ. This is zero at the equator and zero at the North Pole but nonzero in between. So it will not return to its initial direction except at the equator and the Pole.