A limited notion of parallel transport that works on a sphere is, IMO, a good place to start.
One start with the notion that great circles are geodesics of the sphere. (If I'm being pendatic, which is sort of required to avoid having someone point out the oversimplification, I'll add that a great circle is a geodesics of the Levi-Civita connection on the sphere. In the context of GR, use of the Levi-Civita connection is almost always implied, so one often omits specifying it.. And it's useful to note that these great circles are the paths of shortest distance between two points on a sphere (assuming the points aren't antipoidaal. On a sphere, this doesn't affect them being the paths of shortest distance, but things can get messy in general. I'm not going to discuss that further in this post.
This notion of "shortest distance" mostly generalizes to "geodesics of the Levi-Civita connection". But I'm not going to attempt to go into detail.
I'll add one interesting bit of complexity, though. A great circle is different from a course of constant compass heading. The curves of constant compass headings are called loxodromes, and they're not particularly relevant, but - I find them interesting enough to mention, and they serve as an example of something that is NOT a geodesic and not a great circle, but still useful. You can see the wiki entries on
https://en.wikipedia.org/wiki/Loxodromic_navigation, and the related entry on "rhumb lines". I'll even add that one could find a connection that is not the Levi-Civita connection where rhumb lines are the geodesics, though it may need to exclude points on the north and south pole. But that's not really relevant to General Relativity, where we always use the Levi-Civita connection, which also has the fortuitous property of being a path of shortest distance.
The notion of parallel transport along a segment of great circle (geodesic) on a sphere is then simple. You hold a spear, that represents a vector, at a constant angle to your course. You're transporting the spear with you, that's the "transport" part of parallel transport, you're "transporting" a vector along a path.
When you reach the junction of two different segments, you leave the spear in place, as you change course from one segment to the next in your course.
This procedure will let you parallel transport your spear, which represents a vector, along a closed curve on a sphere, and it will be sufficient to show you that the spear winds up pointing in a different direction after you traverse a closed loop.
If you use some spherical trig, you can find that the amount of rotation of the spear is proportional to the enclosed area of a spherical triangle. This generalizes to more complex polygons, though I don't have an accessible proof or reference for this statement. This is related to the notion of the sum-of-angles of a spherical triangle.
This concept of parallel transport generalizes to higher dimensions, and even to curves in space-time (worldlines) rather than curves in space, but probably the generalization isn't so straightforwards, and this short post won't be enough to perform it. At that point, you may need to study a bit of differential geometry. Physics textooks will talk and motivate parallel transport, but they won't go so far as to prove existence or uniqueness. This feeling of lack of p roof of existence and uniquness may be bothersome, all I can say is that there are answers in math if it bothers you enough, and if it doesn't bother you enough you just have to take it on faith that parallel transport exists.