Leaving the details (and the "sales and marketing" arguments made by some enthusiasts for one method in preference to another!) I think the basic point is that there are two ways to construct a mathematical model. One is to consider the behavior of the system at each point, which often leads to an ordinary or partial differential equation. The other way is to consider some properties of a finite (or infinite) part of the system, which often leads to an equation involving integrals.
The advantage of the integral equation approach (when it works - for example it often works better for linear problems than nonlinear ones, as SteamKing's list of BEM applications shows) is that the "dimension" of the solution is often reduced by one, i.e. the solution for the whole region is expressed in terms of the behavior on its boundary. That has obvious advantages if the region in infimite. It can also have disadvantages, if trying to express the solution in terms of the boundary is ill-conditioned for physical reasons, independent of the cleverness of the math (for example the transient behavior of a system after the boundary conditions change from one constant state to a different constant state).
I would say integral equations and the numerical methods derived from them have more limited general utility than differential equations, but they are certainly useful for the right type of problems, as well as being interesting mathematics independent of particular applications.