Yes and no. Your body's immune system is able to help fight abnormally dividing cells by using killer T-cells. For example, in order to maintain the normal function of the body tissues, the killer T-cells can recognize these abnormal cells and tell these cells to commit programmed cell suicide (apoptosis). Some cancer therapies (immunotherapy) rely on boosting the body's immune response to help slow the progression of cancer.
However, most cancer cells eventually become resistant to apoptosis and downregulate antigens so that the immune system can no longer recognize them. So, eventually most abnormally dividing cells will be able to escape control of the immune system and become full-blown cancers.