The term "matrix element" is used by people in quantum theory to name more different, but similar things. The basic meaning is that the matrix element is one number from a set of numbers that serve as numeric representation of some operator ##\hat{A}## in some basis. In your example,
$$
A_{ij} = \langle i | \hat{A} | j\rangle,
$$
is indeed a matrix element of a finite or countably infinite matrix A, in the basis consisting of vectors |i>. This matrix is numerical representation of the operator ##\hat{A}##.
But the term is used also for the numbers that do not belong to these two categories, if the collection of them at least somehow reminds of a matrix. For example, in some papers/books, when you encounter the integral of the form
$$
I(q) = \int K(q,q') f(q') dq',
$$
the authors sometimes refer to the function ##K(q,q')## as to a ##matrix## (continuously indexed), because it reminds of the matrix multiplication, as in
$$
I_i = \sum_j K_{ij} f_j.
$$
In calculating the matrix elements, some basis always has to be chosen, and it so happens that the resulting matrix elements are generally dependent on this basis.
Special elements are the diagonal elements:
$$
A_{ii} = \langle i | \hat{A} |i\rangle.
$$
The expression
$$
\langle \psi |\hat{A}|\psi\rangle
$$
looks similar, but it is not usually called matrix element. It is usually called average or expectation value of the quantity A in "state" ##\psi##. The reason is that it is not frequent to use symbol ##\psi## for a member of a basis; the letter ##\Phi_i## or ket ##|i>## is much better for this purpose.
Nevertheless, ##\langle \psi |\hat{A}|\psi\rangle## can be called matrix element of ##\hat{A}## if really needed, provided ##\psi## is an element of some basis. It is then one of the diagonal matrix elements in that basis.