This was the selling point. This was in the preface introducing the points that Galileo intended to discuss. In actuality, claiming this was intended as an insult to the Pope was pretty thin, but if enough people believed it was an insult, then the Pope had to act on it, whether he believed it to be an insult or not. The Pope had problems of his own and standing up for Galileo would have just made his own problems worse. In other words, the church really did do Galileo wrong.
Who knows what the motivation for persecuting him was. Different members of the church had different views on celestial mechanics - a Copernican model, a Ptolemaic model, and Tycho Brahe's model (that said the Sun and Moon orbited the Earth, while the planets orbited the Sun). There were legitimate arguments for and against each at the time, but the arguments tended to be as much personal attacks on people holding opposing view points as they were logical discussions.
In any event, the actual offense Galileo was found guilty of wasn't as important as finding him guilty of something, with his advocacy of the Copernican model being only one of his offenses.
Defending himself with physical evidence or logic really would have been inadequate for his era. He really needed his allies to mount more substantial personal attacks on his enemies and that involved risk. Personal attacks were not only decided by reputation, but winning these helped a person's reputation while losing these damaged a person's reputation. It wasn't an age dominated by science.