Find gravitational potent. energy - isotropic distribution

[Cocoleia]

Homework Statement

I am told that the gravitational force of a mass m located inside an isotropic distribution of spherical radius R and total mass M is given by
Fg = -GmM(r)/r^2
where r is the distance between m and the center of distribution and M (r) is the mass contained below the distance r (weight between 0 and r). Suppose the sphere is homogeneous of constant density
ρ= M/((4πR^3)/3)
I need to find the gravitational potential energy inside the distribution as a function of M, m, R, r and G

Homework Equations

The Attempt at a Solution

So far I have found the gravitational potential energy, but I don't know how the density is going to come into play. I also don't understand what the isotropic distribution is going to change in my answer and derivation.

 

[eys_physics]

I recommend that you read the Wikipedia article https://en.wikipedia.org/wiki/Newton's_law_of_universal_gravitation, especially the section "Gravitational field". It seems like the formula you are using is not correct, since it is valid for $r>R$ ($R$ being the radius of the sphere). Density is defined as the mass per volume.