That is an interesting remark, I forgot about the delta potential. Just so I don't sound like I'm pulling things out of back of my brain, I'll refer you to a discussion on bound states in Cohen Tannoudji vol 1. complement ##M_{III}##. I'll sum up the discussion here, just listed it in case you want more details that what could be put in one post.
Basically, we look at time-independent Schrodinger equation in one dimension for some solution ##\phi(x)##, and arbitrary potential ##V(x)##. In the first part of the discussion it is proven that bound states always happen at negative energies and the energy spectrum is discrete. I will focus on the discussion on ground state. Acting with the conjugate of the solution and integrating, we get the following form:
$$\int_{-\infty}^{\infty} dx\left( \phi^{*}(x)\frac{d^2}{dx^2}\phi(x) + V(x)\vert \phi(x) \vert^2 \right)= E$$
Using partial integration and property of state functions that they are defined to be square-integrable, you find that the first term on the left, which we can call mean kinetic energy is nonnegative, while the second term is mean potential energy. So we have:
$$E = \langle T \rangle + \langle V \rangle$$
Now we're looking for the lower boundary of energy in order to find the ground state. If potential has a minimum, then we can make the expectation value of potential strictly bigger than that minimal value, using normalization of the states. However, if it has no minima, then we can't find a minimal energy here.
So we conclude, bound states exist in attractive potentials, and they have discrete energy levels, but they don't have to have a ground state, in a sense, such a system would be unstable since under external perturbation it could fall deeper and deeper into the potential well.
Now about the Dirac delta, that function isn't treated in the reference I gave you, since they only treat bound potentials extensively. The reason why Dirac delta allows only one bound state, is because of normalization condition on the state. That is, the zero width of this distribution amounts to such a conclusion, because normalization directly correlates the energy of the bound state to the coefficient in front of delta potential. Potentials of non-zero width wouldn't amount to such an effect, since the states could be properly normalized at different energies, not just one. If you look at Dirac delta as a limit of some sequence of distributions, for example Gaussian or Lorentzian, or even box distributions, you can track how this effect comes about as you push the width of the distribution to zero.
So I would still say I'm certain that in your case you don't have a bound state of minimum energy, aka ground state in this kind of system. When I get time(unless you figure it out yourself), I'll look into that Schrodinger equation to try to create something like annihilation and creation operators for it. Action of annihilation operator on that state would also show if it is a ground state, though finding such operators in this case might be a troublesome thing to do.