Well, solving differential equations is an art. You just have to practice it to find solutions. There is of course a lot of work done by the mathematicians to provide general theorems about their structure and that of their solutions.
In your case of circuit theory or the harmonic oscillator you have a particularly nice class of differential equations, namely a linear differential equation. Here, it's even the kind with constant coefficients. For the undamped harmonic oscillator it reads
[tex]\ddot{x}+\omega^2 x=0.[/tex]
Here the standard ansatz to find a solution is
[tex]x(t)=A \exp(\lambda t).[/tex]
Plugging this into the equation you find
[tex]\lambda^2+\omega^2=0.[/tex]
There are two solutions for [itex]\lambda[/itex], namely
[tex]\lambda_{1/2}=\pm \mathrm{i} \omega.[/tex]
Now the mathematicians have proven that any solution is given as the superposition of two linearly independent solutions, and these we just have found! So the general solution is
[tex]x(t)=A_1 \exp(\mathrm{i} \omega t) + A_2 \exp(-\mathrm{i} \omega t).[/tex]
The constants [itex]A_1[/itex] and [itex]A_2[/itex] are determined by giving initial values, e.g., position and velocity at [itex]t=0[/itex].