Jimster41 said:
The existence of the non-trivial infinitesimal symmetries [...] present a problem for the notion of the co-variant derivative (in other words the existence of the Cevi-Levita connection)?
Good that you ask a question!
As you already suspected, the answer to this question is: No. But that's what questions are for.
It sounds like your question originates in conflating the term "covariant phase space" with the term "covariant derivative". These are completely different concepts. The adjective "covariant" is used for lots of things in lots of situations. Not all concepts that come with the word "covariant" are related.
For the record, I'll say again what the issue is regarding the covariant
phase space in the case of gauge theories:
If gauge symmetries are present, in the sense discussed in the entry, then specifying initial value data (i.e "canonical coordinates and canonical momenta") for field histories on a spatial slice (meaning roughly "at some instant of time") cannot uniquely fix the field histories in the past and future. This is because with anyone field history that has the given initial values, also any gauge transformation of that field history away from that spatial slice will have the same.
Now the "covariant phase space", if it exists, is the space of all field histories that solve the equations of motion, parameterized by the initial value data of these field histories on good spatial slices (Cauchy surfaces). If this exists, it comes equipped with a Poisson bracket, and this is what controls the quantum theory ("canonical commutation relation between coordinates and momenta"). If the covariant phase space does not exist, then there is no Poisson bracket, and hence no quantum theory.
In the presence of gauge symmetries, we just saw that initial value data for field histories on spatial slices does
not serve as parameterization for all on-shell field histories, and hence in this case the covariant phase space does not exist, and hence the quantum theory does not exist.
For this reason, if there are gauge symmetries than one needs to re-think before one can pass to the quantum theory. Namely one has to find a
different field theory which
a) does have a covariant phase space;
b) is still equivalent to the original field theory in a suitable sense.
This re-thinking is called
BV-BRST gauge fixing, discussed in chapter
12. Gauge fixing.