Using energy and momentum conservation, compute the 4-vector of the particle, and it's Lorentz square is the square of the mass.
Say for instance you detect the decay products of the particle, add up all 4-vectors, the square of the mass is the difference between (1) the square of the total energy of the decay products and (2) the square of the total momentum carried by the decay products.
You could also do it if by any chance you can assume the particle is the only one missing in a certain balance, and computing the missing 4-vector.