Consider someone on the top floor of a 400 floor building, and someone on the ground floor.
If they both have the same human body and are going to push 100kg desks, then they both have the same potential energy to do work on these desks, regardless of their gravitational potential energy with respect to each other. The way you asked your original question about their capacity to do work while being at different heights, and your implication that you wonder if potential energy is not real lead me to think that your contention is that the increase in gravitational potential energy gives the person a higher absolute potential energy with respect to the objects that they might do work on, as in your example objects on the same floor as the person. If that were true, which it isn't, then it would make sense for you to question if potential energy is not real since this is not what we experience.
Just because one person has a higher gravitational potential energy does not mean that that potential energy is relevant when you compare two people's ability to do work. When you talk about their capacity to do work, such as pushing a desk, you are using 2 different potential energy references for the two cases, and so it does not make sense to say a person on a higher floor has more potential energy when any work they do on that floor will be at the same gravitational potential energy, and thus cancel out when compared to someone on a different floor in the building.