There is quite a lot of black magic talked about with waveguides, because the waves will go through any old copper pipe. However, a rectangular WG has the advantage that there is only one mode, so it avoids multiple signals which can be a problem for high bandwidth transmission of data or TV. It is also difficult to use ordinary pipe bends because a "proper" waveguide will have carefully chosen bending radius and will often have tuning screws to reduce unwanted reflections. I cannot see any problem in principle about using circular tubing, although more modes may spring up and polarisation may rotate due to imperfections. One issue is that it is hard to make WG to coax transitions, so that is why peope buy them and then stick to standard sizes. Incidentally, for 50 years it has been the practice for microwave links to use corrugated elliptical waveguide, which can to some extent be bent like a cable.