Your observation is correct so far. The same way we get [x,p]=i from the definition p=-id/dx, we can get [t,H]=-i from the definition H=id/dt. But you need to think about what sort of functions these operators are supposed to act on. To get [x,p]=i, you have to assume that x and p act on functions of x, and to get get [t,H]=-i, you have to assume that t and H act on functions of t. To get both, you have to assume that all these operators are acting on functions of both t and x. At first glance, this doesn't look like a problem since something called \psi(\vec x,t) appears in the Schrödinger equation.
To see why it is a problem, we must look at the uncertainty relation:
\Delta A\Delta B\geq\frac 1 2|\langle[A,B]\rangle|
Note that there's an expectation value on the right. The expectation value is defined using an inner product. This means that if we use an inner product on the set of functions of the form \psi:\mathbb R^4\rightarrow\mathbb R, we can derive both [x,p]=i and [t,H]=-i. But what inner product do we actually use in QM? It looks like this:
\langle f,g\rangle=\int_{\mathbb R^3} f(\vec x)^*g(\vec x) d^3x
As you can see, it's defined for functions of 3 variables, not 4. This means that the right-hand side of the uncertainty relation makes sense when A=x, B=p, but not when A=t, B=H.
This is why "time is not an observable in QM".
However, there is an energy-time uncertainty relation that makes sense in QM. Use the first link in George's post. In that one \Delta t is the time it takes the expectation value to change by one standard deviation.