Of course not! Wave functions can provide only probabilities if their modulus squared is integrable, i.e., you can choose the normalization such that
$$\int_{\mathbb{R}} \mathrm{d} x |\psi(x)|^2=1.$$
A "position eigenstate" is not a proper function but a distribution (in the sense of "generalized function"), i.e., you cannot even square it, and the expression ##\delta(0)## makes no sense at all. It's undefined.
It's purpose is to map a function in a certain dense subspace, which defines the part of the Hilbert space of square integrable function, where the position (and also the momentum) operators are defined, and for these functions you have
$$\int_{\mathbb{R}} \mathrm{d} x' \delta(x-x') \psi(x')=\psi(x).$$
It's also true that the generalized position eigenstates (where again I used the word "generalized" in the sense of distributions) obey
$$\langle x|x' \rangle=\delta(x-x').$$
The same holds for momentum eigenstates
$$\langle p|p' \rangle=\delta(p-p').$$
You also may know that the generalized momentum eigenstates in the position representation read
$$u_p(x)=\langle x|p \rangle=\frac{1}{(2 \pi \hbar)^{1/2}} \exp \left (\frac{\mathrm{i} x p}{\hbar} \right).$$
Again this is not a square-integrable function, i.e., you cannot normalize it since ##|u_p(x)|^2=1/(2 \pi \hbar)##, which is not integrable over the entire real line, but still you can use the completeness relation in the form
$$\int_{\mathbb{R}} \mathrm{d} x |x \rangle \langle x| = \int_{\mathbb{R}} \mathrm{d} p |p \rangle \langle p |=\hat{1}.$$
This implies that you can write any proper wave function in the domain, where posision and momentum operators are defined either in position and momentum representation, via
$$\psi(x)=\langle x|\psi \rangle=\int_{\mathbb{R}} \mathrm{d} p \langle x|p \rangle \langle p|\psi \rangle=\int_{\mathbb{R}} \mathrm{d} p \frac{1}{(2 \pi \hbar)^{1/2}} \exp \left (\frac{\mathrm{i} p x}{\hbar} \right) \tilde{\psi}(p),$$
which is just a Fourier transform of the momentum-space wave function. Also the inverse follows immediately from this formalism
$$\tilde{\psi}(p) = \langle p|\psi=\int_{\mathbb{R}} \mathrm{d} x \langle p|x \rangle \langle x|\psi \rangle=\int_{\mathbb{R}} \mathrm{d} x \frac{1}{(2 \pi \hbar)^{1/2}} \exp \left (-\frac{\mathrm{i} p x}{\hbar} \right) \psi(x),$$
which is also well known from the theory of Fourier transformations.
