It depends on first of all being able to estimate the mass of the STAR that the planet is orbiting
The mass of a normal star can be estimated by it spectrum (its colors of light). More massive stars burn HOTTER and bluer (shorter wavelengths).
Logically you might first ask how people discovered and calibrated this relation between color and mass. they used binary pairs of stars, that orbit their common center of mass. if you know the mass of one star (say it is the same color as the sun, which we know the mass of) and you can watch them go around each other, then you can tell the mass of the other. then you measure the color of the other and that gives another data point. Information about how color and mass are related builds up.
Once you know the mass of the star you can figure out the mass of the planet by how much it causes the star to wobble too and fro. the more massive star takes a more massive planet to make it wobble a certain amount.
=========================
The wobble method is just one of several methods to detect planets. Let's focus on that. In that case you don't even have to SEE the planet. You just see the star approaching, say at 10 meters per second (max), and after a while you see it slow down and start receding, at 10 meters per second (max). And then approach, and then recede, and so on. Let's assume for simplicity that we are looking at the orbit plane edge on, so we see all the to and fro speed.
Maybe someone else will take over and explain this in more detail, with Kepler's law etc. It's after midnight here and I'm too sleepy to continue.
anyway, good question! The PERIOD of the wobble gives away the distance between the two bodies (Kepler). If you know the distance, you can relate the speeds to the masses. Gotta sleep.