The turning on of the voltage supply sets a new equilibrium point of reference for the oscillation of the capacitor voltage that's 1V above ground (assumed to be the negative lead of the voltage source).
To make a mechanical analogy, suppose you had a mass hanging on a spring, but had the ability to switch gravity off and on. Starting with gravity off, the mass sits at rest at the end of the spring which is at its un-stretched, natural length. There is no motion, and we set the zero reference for position where the mass is, and take any downward displacement to be positive (just a choice of reference point and axis direction).
Now you suddenly switch on gravity. The new equilibrium position for the mass would be below the old one; it's where the mass would come to rest if there were damping forces involved. Gravity will pull the mass down through this new equilibrium position (where it will have maximum velocity), until it finally slows and then turns around at a distance below the new equilibrium point equal to the distance of the equilibrium point from the original rest length of the spring. The mass will continue to oscillate around the new equilibrium point, reaching only as high as the old equilibrium position (zero displacement); the displacement will never go negative.
So the equation for the displacement becomes A*(1 - cos(ωt)) where A is the amplitude of the oscillation, equal to the displacement of the new equilibrium position to the old equilibrium position).
The same thing is happening with the LC circuit. If there were a bit of resistance in the circuit, the capacitor would "come to rest" with a potential difference of 1V across it. That's its "new equilibrium position" around which it oscillates.
Yes, it's a sort of filter. Replace the DC voltage source with a variable frequency signal generator and you will see a peak in current when the signal generator matches the resonance frequency. With no resistance in the circuit the signal generator will continue to feed energy to the oscillations, in phase, so the voltages and currents will grow without bound! Away from this "selected" frequency the response is not so dramatic.