First of all, when it comes to unbound operators hemitizity is not enough, you need self-adjointness, but this is a finer mathematical point. I only say this that you are aware of that the following is the usual sloppy physicists' use of mathematics.
It is most simple to start in a concrete representation. In the case of non-relativistic quantum theory it's most easy to start with the position and momentum representations. In the position representation a pure state can be represented by a square-integrable complex-valued function ##\psi(t,x)## (just take one-dimensional motion as the most simple example. Then ##|\psi(t,x)|^2## is the probability distribution for the position of the particles at time ##t##.
To understand why the observables are represented by the operators you learn as the next step, you have to remember how they can be defined in a quasi algebraic way in analytical mechanics. For this you need the Hamilton formalism in the Hamiltonian formulation, i.e., in your case of one-dimensional motion of a particle you have a phase space with a position coordinate ##x## and the canonical conjugate momentum ##p##. The algebra in classical physics is the algebra built by Poisson brackets. For ##x## and ##p## you have ##\{x,p\}_{\text{pb}}=1##. From the point of view of symmetry transformations this tells you that momentum is the infinitesimal generator of spatial translations, and as such you define it also in quantum theory. As a heuristic principle you can thus use "canonical quantization", i.e., you make the classical observables to self-adjoint operators on Hilbert space, and instead of the Poisson brackets you use commutators via the "translation description": ##\{A,B \}_{\text{pb}} \mapsto \frac{1}{\mathrm{i} \hbar} [\hat{A},\hat{B}]##.
Now it's obvious how to define the operators in position representation: Obviously position is represented by multiplying the wave function with its argument ##x##:
$$\hat{x} \psi(t,x)=x \psi(t,x).$$
Now you need another operator for momentum which satisfies the canonical commutator relation
$$\frac{1}{\mathrm{i} \hbar} [\hat{x},\hat{p}]=\hat{1}.$$
It's very easy to see that
$$\hat{p}=-\mathrm{i} \hbar \partial_x$$
fulfills this commutation relation, because for any wave function (that is differentiable!) you have
$$\hat{p} \hat{x} \psi(t,x)=-\mathrm{i} \hbar \partial_x (x \psi(t,x))=-\mathrm{i} \hbar [\psi(t,x)+x \partial_x \psi(t,x)]=-\mathrm{i} \hbar \psi(t,x) + x \hat{p} \psi(t,x),$$
i.e., indeed
$$[\hat{p},\hat{x}] \psi(t,x)=(\hat{p} \hat{x}-\hat{x} \hat{p}) \psi(x)=-\mathrm{i} \hbar \psi(t,x).$$
Since this is valid for any wave function, for which the momentum and position operators are well defined, you indeed have
$$[\hat{p},\hat{x}]=-\mathrm{i} \hbar \hat{1}.$$