Again one should really stress in introductory lectures the difference between a vector, which is a basis-independent object. In physics it describes real-world quantities like velocities, accelerations, forces, fields like the electric and magnetic field or current densities etc, and components of the vector with respect to some basis. It's a one-to-one mapping between the vectors and its components given a basis.
In quantum theory the kets are vectors in an abstract Hilbert space (with the complex numbers as scalars). In non-relativistic QM with finitely many fundamental degrees of freedom (e.g., for a free particle position, momentum, and spin) the Hilbert space is the separable Hilbert space (there's only one separable Hilbert space modulo isomorphism).
Then there are linear forms on a vector space, i.e., linear maps from the vector space to the field of scalars. These linear forms build a vector space themselves, the dual space to the given vector space. In finite-dimensional vector spaces, given a basis, there's a one-to-one mapping between the vector space and its dual space, but not a basis-independent one. This changes if you introduce a non-degenerate fundamental form, i.e., a bilinear (or for complex vector spaces sesquilinear) form, where you get a basis-independent one-to-one-mapping between vectors and linear forms.
For the Hilbert space, where a scalar product (sesquilinear form) is defined you have to distinguish between the continuous linear forms (continuous wrt. to the metric of the Hilbert space induced in the usual way from the scalar product) and general linear forms. For the latter there's a one-to-one correspondence between the Hilbert space and its ("topological") dual, and in this way these two spaces are identified.
In QM you need the more general linear forms since you want to use "generalized eigenvectors" to describe a spectral representation of unbound essentially self-adjoint operators. This always happens when there are such operators with continuous spectra like position, momentum. One modern mathematically rigorous formulation is the rigged Hilbert space. There you have a domain of the self-adjoint operators like position and momentum, which is a dense sub-vector-space of the Hilbert space. The dual of this dense subspace is larger than the Hilbert space, i.e., it contains more linear forms than the bound linear forms on the Hilbert space.
Using a complete orthonormal set ##|u_n \rangle## you can map the abstract vectors to square-summable sequences ##(\psi_n)## with ##\psi_n =\langle u_n|\psi \rangle##. These sequences build the Hilbert space ##\ell^2##, and you can write the ##(\psi_n)## as infinite columns. The operators are then respresented by the corresponding matrix elements ##A_{mn}=\langle u_m|\hat{A}|u_n \rangle##, which you can arrange as a infinite##\times##infinite matrix. This is a matrix representation of QM, and was the first way how modern QM was discovered by Born, Jordan, and Heisenberg in 1925. The heuristics, provided by Heisenberg in his Helgoland paper, was to deal only with transition rates between states. Heisenberg had the discrete energy levels of atoms in mind but could demonstrate the principle first only using the harmonic oscillator as a model. Born immediately recognized that Heisenberg's "quantum math" was nothing else than matrix calculus in an infinite-dimensional vector space, and then in a quick sequence of papers by Born and Jordan, as well as Born, Jordan, and Heisenberg the complete theory was worked out (including the quantization of the electromagnetic field!). Many physicists were quite sceptical about the proper meaning of the infinitesimal vectors and matrices.
Then you can also use the generalized position eigenvectors ##|\vec{x} \rangle##, which leads to the mapping of the Hilbert-space vectors to square-integrable (integrable in the Lebesgue sense) functions, ##\psi(\vec{x})=\langle \vec{x}|\psi \rangle##. This is the Hilbert space ##\mathrm{L}^2## of square-integrable functions, and the corresponding representation is the 2nd form modern quantum theory has been discovered in 1926 by Schrödinger and is usually called "wave mechanics". Schrödinger very early has shown that "wave mechanics" and "matrix mechanics" are the same theory, just written in different representations. In Schrödinger's formulation, heuristically derived from the analogy between wave and geometrical optics in electromagnetism, the latter being the eikonal approximation of the former. Schrödinger used the argument backwards, considering the Hamilton-Jacobi partial differential equation as the eikonal approximation of a yet unknown wave equation for particles, which was just the mathematical consequence of the ideas brought forward in de Broglie's PhD thesis, which was favorably commented by Einstein as a further step to understande "wave-particle duality".
Almost at the same time Dirac came with the now favored abstract formulation, introducing q-numbers with a commutator algebra heuristically linked to the Poisson-bracket formulation of classical mechanics. This was dubbed "transformation theory", because the bra-ket formalism enables a simple calculus for transformations between different representations like the Fourier transformation between the position and the momentum representation of Schrödinger's wave mechanics.
Finally the entire edifice was made rigorous by von Neumann, recognizing the vector space as a Hilbert space and formulating a rigorous treatment of unbound operators. The physicists' sloppy ideas could then be made rigorous by G'elfand et al in terms of the "rigged Hilbert space" formalism. For the pracitioning theoretical physics you can get quite well along without this formalism, though it's always good to know about the limitations of it, and it's good to know at least some elements of this formalism. A good compromise between mathematical rigorousity and physicists' sloppyness is Ballentine's textbook.