I'm just not convinced that it should be completely dismissed as worthless, though. It could be that it has an audience problem, in terms of being too abstract for beginners, but with a lot of superfluous, elementary junk to wade through for the people who might gain something from it. Specifically, if anyone is crazy enough to want to learn topos theory, it appears to be a good place to start.
Here's what Baez said about it:
"It may seem almost childish at first, but it gradually creeps up on you. Schanuel has told me that you must do the exercises - if you don't, at some point the book will suddenly switch from being too easy to being way too hard! If you stick with it, by the end you will have all the basic concepts from topos theory under your belt, almost subconsciously."
So, I don't think the point is just abstraction for its own sake. I don't know topos theory, so I'm not sure how much of a point there is to it (in fact, it seems like too abstract nonsense to me, but what do I know?), but it does appear to be something that some logicians like, so it might be eventually relevant to the OP, but it sounds like that will be much later down the line.