I am not sure it is so simple. Your logic works very well if we are on a flat manifold, so you can define some sort of Carthesian coordinates, and then relative to those coordinates you can define other coordinate systems. The Christoffel symbols will then arise naturally.
What happens if you are not on a flat manifold (##\mathcal{M}##)? In this case there may be no single coordinate system that covers the manifold fully. That's not a problem. You can cover different patches of the manifold with different coordinate systems. Then at every point (##p\in \mathcal{M}##) on the manifold you can define a different tangent vector space (##T\mathcal{M}|_p##). However, now it is not trivial how you go from one point to another (##T\mathcal{M}|_{p_1}\to T\mathcal{M}|_{p_2}##). Unless you can define this you cannot differentiate vector fields. So you need to connect vector spaces in some way - that's when you need the connection (##\Gamma^\alpha_{\beta\gamma}##). There is more than one way to define a valid connection, if you use metric tensor to define the connection, then you get Christoffel symbols, but there are other ways too.
I think it is treated very well in Lovelock & Rund "Tensors, Differential Forms, and Variational Principles" (Sec 3.3). Another favourite is Bachman "Geometric approach to Differential Forms"