PeroK said:
The dual space is the space of all linear functionals. And it can be shown that every linear funtional is the inner product corresponding to some vector as above. I.e. the dual space is the set of these functionals defined by the inner product.
Here one must be a bit careful. The Hilbert space is a pretty special vector space. By definition it is a metric vector space with the metric induced by the scalar product, and you have to distinguish between different kinds of dual spaces used in quantum mechanics.
Within the Hilbert space itself there is the the dual in the restricted sense of continuous linear forms, and these build a vector space isomorphic to the Hilbert space itself, i.e., for each such linear form there's a (true normalizable) vector ##|\psi \rangle## such that the linear form is given by the adjoint, i.e., ##\langle \psi|##.
Nevertheless QT doesn't work with this kind of linear forms only, but as soon as you have to describe observables by an unbound essentially self-adjoint operator, and this is already the case for the most simple case of a single classical particle quantized by "canonical quantization", i.e., defining position and momentum operators obeying the canonical commutator relations, defining the Heisenberg algebra. They are only defined on a dense subspace of the Hilbert space. As self-adjoint operators this domain must also be their co-domain. This "nuclear space" is a proper subspace, and now you also have to consider the linear forms defined on this restricted subspace, defining its dual, and this dual is larger than the dual of the Hilbert space. This allows you to handle also "generalized eigenstates" of a self-adjoint operator in a formal manner, and this enables constructs like the eigenvectors of the position operator ##|\vec{x}' \rangle## and their (formal) dual. In position representation you explicitly see that this implies the use of generalized functions (distributions), in this case the Dirac ##\delta## distribution, ##u_{\vec{x}'}(\vec{x}) =\langle \vec{x}|\vec{x}' \rangle=\delta^{(3)}(\vec{x}-\vec{x}')##. This is of course not defined on the entire Hilbert space of square-integrable functions (which is the concrete realization of the Hilbert space in terms of the position representation, which is nothing else than "wave mechanics") but only on the corresponding set of test functions (like the Schwartz space of "rapidly-enough vanishing functions").
All this hand-waving can beformalized mathematically in terms of the socalled "rigged-Hilbert space formalism", which is much closer to the physicists "robust mathematics" invented by Dirac than the equivalent more conventional formalization of it by von Neumann.
For a gentle introduction see
L. E. Ballentine, Quantum Mechanics, World Scientific,
Singapore, New Jersey, London, Hong Kong (1998).
For a more thorough treatment see
A. Galindo and P. Pascual, Quantum Mechanics, Springer
Verlag, Heidelberg (1990), 2 Vols.
of de la Madrid's PhD thesis:
http://galaxy.cs.lamar.edu/~rafaelm/webdis.pdf