Interesting question. There are plenty of bad books out there, but I hope I am unaware of most of them. Here are a few controversial choices among math books: many people like these, but I consider them horrible and ill-written:
* Folland's Real Analysis - many people complain that Rudin's analysis books are slick, dense, and unmotivated. Folland's is non-slick, dense, unmotivated, and full of typos.
* Hatcher's Algebraic Topology - one of a small handful of books I have ever put in a fireplace
* Dummit and Foote's Abstract Algebra - this book is competent and has a wonderfully broad coverage of material, but it is by far the most boring algebra book I've ever read, and it somehow sucks all the joy out of what is in fact a beautiful subject. If this were my only exposure to modern algebra, I would hate the subject.
* Euclid's Elements - of course this is a monumentally important book, and an amazing achievement of human thought. But it is full of maddening "definitions" such as "a point is that which has no part", and many of its proofs depend on unstated assumptions and/or inferences from diagrams which do not fully capture the general case of the theorem statement. With a companion book such as Hartshorne's "Geometry: Euclid and Beyond", it might be worthwhile reading, but worthwhile does not necessarily mean that it won't be a miserable experience.
I also found Halliday and Resnick's "Physics" (an early '80s edition) to be awful. Apparently newer editions are even worse.
One could probably list almost any textbook aimed at college freshmen or lower.
But undoubtedly many people find something of value in all of these books, so I would not go so far as to recommend that no one should read them.