I actually never read about it, so I can't make any claims about it's significance.
Your last phrasing of the problem I believe doesn't help much. You are given a group G, and a non-trivial subset H, finding an f that is the identity when restricted to H is trivial. Just let f be the identity map!. But I'm not quite sure that is what you had in mind.
Now, let's try another way, let's say you have a group G, an endomorphism f and a non-trivial subset H such that x in H implies f(x) = x. Is H the only subset, or the largest subset with this property?
To answer this, we may investigate what the set of all such elements would look like?
Define H as a subset of G that contain the elements of G such that x in H implies f(x) = x.
H is non-empty and contains the identity, as shown earlier.
H is closed under the binary operation: assume there is a non trivial x that is preserved by f, i.e f(x) = x in H. Then we know that f(xx) = f(x)f(x) since f is a homomorphism. But we must have f(xx) = f(x)f(x) = xx. So xx also satisfy this property and are in H.
H contains the inverse of each element: 1 = f(1) = f(x x^{-1}) = f(x) f(x^{-1}) = x f(x^{-1}), so that f(x^{-1})= x^{-1}
Hence you may conclude that H is a group (cyclic subgroup if we picked only x), and the restriction of f to H is a homomorphism(injective). It is also the smallest subgroup of G that has f(x) = x for some x in G.
If G is generated by a set X, and the endomorphism f preserve each element in that set, i.e f(x) = x for each x in X, then f is the identity map!
So you may conclude from this discussion that if H is a subset of G, where f(x) = x for all x in H, then there's a subgroup K of G generated by H such that f(K) = K.Does this help?