"Virtual particles" is a quite misleading term of a mathematically well defined object, known as the Feynman propagator in relativistic
perturbative quantum field theory. Also the idea that Feynman diagrams depict scattering processes as if you could think about them like collisions of miniature billiard balls is quite misleading. One should keep in mind their quantum (field) theoretical meaning.
They depict in a very clever way formulas that allow you to systematically calculate S-matrix elements for scattering processes in quantum field theory. The external lines depict asymptotic free states of the incoming and outgoing particles, usually plane-wave momentum eigenstates (which are distributions rather than functions by the way). These states can be identified as specific kinds of particles (say electrons) hitting a detector with a quite sharp momentum and can be counted to get measure a cross section for some process of interest (e.g., elastic electron-electron scattering), which is evaluated in QFT using the S-matrix elements which are written cleverly in terms of Feynman diagrams.
The internal lines stand for propagators. These do not symbolized particles that can somehow be detected in the above sense with real-world detectors. They are just mathematical objects used to evaluate the matrix elements.
To really understand elementary particles you have to study quantum field theory and see how the Feynman rules are derived and which meaning the physical quantities have you can define from them. The Feynman diagrams should be seen as a very clever symbolism to write down complicated formulae rather than pictures of what's going on in real-world scattering processes.