Start with [itex]Ax^2+ Bxy+ Cy^2[/itex] and complete the square just like with the single variable problem:
[tex]A(x^2+ (B/A)xy+ (By/2A)^2- (By/2A)^2+ Cy^2= A(x- By/2A)^2+ y^2(C- (B/2A)^2[/tex]
Assuming A is positive (which you could guarantee by multiplying the entire equation by -1 if necessary), that will be a "sum of squares", and so the graph is an ellipse or circle, if and only if [itex]C- B^2/4A^2> 0[/itex] or [itex]B^2- 4AC< 0[/itex].
It will be a "difference of squares", and so the graph is a hyperbola, if and only if [itex]B^2- 4AC> 0[/itex] and will have only one square, and so the graph is a parabola if and only [itex]B^2- 4AC= 0[/itex].