Nice work on the 4-vector approach, the calculation was a lot simpler when you keep E1,E2,P1,P2 as symbols.
Anyway, onto inelastic collisions. An inelastic collision just means that v1=v2 after the collision.
The important facts about systems of particles are these
1) The total energy of all the particles involved add
2) The total momentum of all the particles involved add
3) The mass (invariant mass) m of any specific system or subsystem of particles is given by m^2 = E^2 - p^2 in geometric units, where E is the energy of the system or subsystem, and p is its momentum.
Masses don't necessarily add, as can be seen from the defintion.4) As long as the system is isolated (not interacting with something else), the invariant mass of the system will be invariant. This can be shown with a particle-swarm model, if and only if every particle interacts with other particles only when they are at the same point. Particles that interact with each other via long-range forces need more attention.
If the particles interact with each other via fields, one has to include the energy and momentum of the fields into the system to get a coherent picture in which energy and momentum remain conserved. One might also be able to picture the field interactions as occurring by "virtual" particles to get the same result.It's not particularly relevant to this problem, which is an isolated system but an example of a problem where a non-isolated system gives a non-invariant mass E^2-p^2 can be found
https://www.physicsforums.com/showthread.php?t=117773
Another way of describing the issue here - the system m3 has relativistically significant amounts of heat energy, so one needs a theory of relativistic thermodyamics to model the system m3 in detail.
One simple paper that I've read recently that I like is
http://arxiv.org/PS_cache/physics/pdf/0505/0505004.pdf
but I really haven't read extensively about relativistic thermodynamics, there may be better approaches out there that I'm not aware of.
The bottom line is that the answer to this problem is OK, but the problem actually involves more advanced elements that are being glossed over to get a simple answer.