All mammals do according to this: http://www.madsci.org/posts/archives/may97/863469598.Zo.r.html Quite why you can't find (what I presume to be) your cat's, I don't know. Come to think of it, I've never noticed my guinea pigs' either.
Hi, Cats, guinea pigs are mammals so they have one. Dolphins have too... http://www.enchantedlearning.com/subjects/mammals/
All placental mammals have an umbilical cord during fetal development. You won't find a clear belly button like humans have (it might be more interesting to ask why humans get such a distinct belly button "scar" that other mammals don't get), but there will be a small, usually flat, scar with lighter coloration than the skin around it. Typically it's hidden under fur.
Most of the time the cord tears off by itself. Animals that birth standing up will tear the cord on the way out/down. I have seen a litter of kittens where the cords stayed attached to several kittens for what appeared to be a couple of days. They were all wrapped up on various body parts and it was hard to cut them apart.