Pardesi,
You have several questions listed involving forces of constraint- I empathize. Let's see if I can help out with the conceptual foundations. I have the second edition of Goldstein, so the equation numbers I use may be slightly different than yours.
The Lagrangian (and Hamiltonian) formulation is not much different than F=ma. It's just written in a more general framework involving energy. The question "constraint forces" tries to address is:
How can we describe constrained motion using F=ma?
For example, picture a ball rolling on a surface (with gravity present). Gravity acts 'down', yet the ball cannot simply fall, it is constrained to lie on the surface. The problem of constraint forces is used to describe the force keeping the ball on the surface.
Remember, this is classical mechanics and was invented a loooooooong time before people knew about electrons, there was no proof of atoms existing, no theory for the electromagnetic (or even electrostatic) forc. So, we posit that a force constrains the ball to the surface.
The ball (eventually) comes to rest somewhere on the surface- an equilibrium configuration. Thus, the sums of forces and torques is zero. Goldstein then writes Eq. 1-40 (page 16) invoking "virtual work", but it's just another way to write that the system is in equilibrium.
He then posits "the net virtual work of the forces of constraint is zero" without giving any real jusitification. But, it's clear that *at equilibrium*, the force keeping the ball in contact with the surface must act normal to the surface, while the ball can only move tangentially on the surface. So the dot product of force and (virtual) displacement is zero.
The Equation 1-43 to 1-47 are straightforward: 1-46 and 1-47 are just equivalent ways of expressing a differential, with the idea that dt can be eliminated. The text between 1-47 and 1-48 is a discussion of coordinate transformations rather than physics.
Does that help?