Virtual particles in the sense I define them - as internal lines in Feynman diagrams - are a mathematical tool used for an approximation - so-called perturbation theory. Unfortunately most quantum field theories like QED, QCD etc. are very complicated and we do not have the mathematical tools to solve them exactly. But fortunately we have several approximations. Perturbation theory is used for weak coupling, where it makes sense to start with free, non-interacting particles and to add small corrections for interactions. Interestingly this works very well in many cases, especially for scattering experiments (but there are other problems like QCD bound states, e.g. protons, neutrons, ... where this approximation is useless).
If you would have mathematical tools to solve quantum field theories exactly, there would be no reason to introduce perturbation theory, there would be no name for the mathematical artifacts, and we would not have these discussions. Before studying QCD I was working on two-dimensional models, fields living on a line = one space dimension + one time dimension. These models a rather simple, a good starting point for beginners. There are exactly solvable models with bound states, there are other approximations, and thefore no reason to use perturbation theory. In QCD there are tools to study non-perturbative aspects, tools like chiral effective theories, lattice gauge theories, ... All these tools do not require Feynman diagrams and therefore - using these tools - there is nothing which we call 'virtual particle'.
In addition there are mathematical reasons against perturbation theory. We know that strictly speaking it is ill-defined, it is something which does not exist mathematically, but nevertheless it seems to work in a very restricted sense. And there are applications where this ill-defined math does produce correct results which agree with experiments (strange, isn't it? we can prove that it does not work, but using it seems to work ...). Now what I am saying is that we can START with a formulation w/o any approximation and w/o virtual particles. Then we have to introduce approximations, but doing this we CREATE several problems, or we apply approximations outside their scope of applicability, so the approximation BREAKS DOWN. Doing this we have mathematical artifacts - virtual particles - but due to the problems we introduce there seems to be no good reason to believe in virtual particles to be more than just limited tools.
Does this help?