It actually does, don't forget you conductor sample is finite. The external electric field is always normal to a conductor's surface. If you insert a rectangular conductor into a uniform field, like it's shown on your drawing, what this field at the upper and the lower surfaces would be?