Your right-hand side doesn't really make sense; operators act on states but on the right-hand side A is trying to act on a number. I think what you want is
##\langle x | A | \psi \rangle = \int dx' \langle x | A | x' \rangle \langle x' | \psi \rangle##
The left hand side is the position-space wave function of the state ##A | \psi \rangle## which the right-hand side gives in terms of the position-space wave function of the state ##| \psi \rangle## and the position-space matrix elements of the operator A.
For example, you can derive the position-space matrix elements of the momentum operator ##p## from the canonical commutation relation ##[x, p] = i\hbar##. You find
##\langle x | p | x' \rangle = i \hbar \delta ' (x' - x)##
Where ##\delta'(x)## is the derivative of the Dirac delta function. Plugging this into the above formula you get
##\langle x | p | \psi \rangle = \int dx' i \hbar \delta ' (x' - x) \langle x' | psi \rangle = -i \hbar \int dx' \delta(x' - x) \frac{d}{dx'} \langle x' | \psi \rangle = -i \hbar \frac{d}{dx} \langle x | \psi \rangle##
which tells you that the position space wave function of ##p | \psi \rangle## is -i hbar times the derivative of the position space wave function of ##| \psi \rangle##