I think "difficult" in the context of singing songs needs to be clarified by splitting it into two different issues.
Some songs are extremely difficult for technical reasons, for example because of the vocal range they require, so that only someone totally comfortable in that range should sing them. This allows for many different (more than once) performances, but greatly restricts who ought to undertake the challenge.
Other songs, which are not necessarily difficult technically, have become popular via such sensitive, energetic, vervy, inspired, or otherwise remarkable performances that no one else should really "be allowed" to sing them because they just can't equal or top the personality and "soul" individual to the performer that made the song popular.
The performer that made them famous has seriously raised the bar on people's perception of how deep and rich a performance constitutes a proper one despite the fact a large percentage of singers might be able to give a performance that would be perceived as very successful in the absence of the famous version. It's not difficult to sing the song, it's difficult to top the performance that made the song famous.