I have in fact encountered two somewhat different definition for the adjoint. This is the other one. When X and Y are norm spaces, we set
[tex]
X^* := \{x^*:X\to\mathbb{C}\;|\; x^*\;\textrm{is continuous and linear}\}[/tex]
and similarly Y*. When T:X->Y is continuous linear mapping, we set
[tex]
(T^* y^*)x = y^*(Tx),\quad\forall y^*\in Y^*,\; x\in X.[/tex]
Notice that this formula defines a linear mapping
[tex]
(T^* y^*): X\to\mathbb{C},\quad (T^* y^*)\in X^*,[/tex]
and thus also a mapping
[tex]
T^*: Y^*\to X^*.[/tex]
If we have a Hilbert space H, and a linear mapping T

->H, then strictly according to the previous definition, we have T*

*->H*. However, the dual of a Hilbert space is isometric to the space itself, so we can identify all vectors of H* as vectors of H, and let the action of [itex]x^*\in H^*[/itex] be given with the inner product. So [itex]x^*(z)[/itex] becomes replaced by (x|z), if the [itex]x^*\in X^*[/itex] and [itex]x\in X[/itex] assumed to be identified in the isometry. The previous condition then becomes
[tex]
(T^* y|x) = (y|Tx),[/tex]
so the old formula, that you knew for matrices already, is obtained as a special case.
Then there is another definition, which deals with unbounded operators. If we have T:D(T)->H, with [itex]D(T)\subset H[/itex] being some subspace, it is possible to give some definitions [itex]T^*:D(T^*)\to H[/itex] so that the same inner product formula is still valid for some vectors, but this time the domain issue gets more difficult.