Your equations are not quite right. It should be that the current through a capacitor is given by:
I = CdV/dt
and the voltage across an inductor is given by
V = LdI/dt
Physical quantities should take on finite values. Having the current or voltage change instantaneously corresponds to infinite dI/dt or dV/dt, which is unphysical. But if you're not convinced, then keep in mind that what these equations are doing is making sure that that can't happen. Let's take an example:
Say you have a battery of voltage V and you connect it to a load having resistance R through a switch. When the switch is open, the current is zero. IF you assume that the circuit has only resistance, then according to Ohm's law, when the switch is closed, the current should be equal to V/R. So, when you throw the switch, the current jumps instantaneously from 0 to V/R? The charges were not moving, and then suddenly they were? That suggests infinite acceleration, which suggests infinite force. But the charges are NOT being acted on by an infinite force (which is an impossibility anyway). So something must be wrong with our model for the system (namely Ohm's law), since it is leading us to a result that violates the laws of physics. The error we're making is that we're failing to consider the small amount of inductance L that this circuit (and indeed any circuit) has. If you include the L with the R, and solve for the current as a function of time i(t), you'll find that when you throw the switch, the current increases in a nice gradual way from zero to its final steady state value of V/R. The faster you try to change the current (i.e. the larger dI/dt is), the larger is the induced voltage or EMF (which is equal to LdI/dt) that *opposes* that change. Hence, the unphysical instantaneous changes cannot occur.
A similar story is true with capacitance. Every circuit has some natural capacitance, meaning you can't change the voltage on a node from zero to some final value instantaneously (like a step function). Instead, it ramps up gradually. Granted, if the capacitance is small, the ramp can be very steep.