I saw a presentation by a neuroscience professor who studies the neuroscience of music (don't remember the name or the venue now) and his (speculative) suggestion was that music was about expectation. Often, when our expectations are met in a timely manner, we are satisfied. A musical rhythm gives you an opportunity at every measure to have your expectations met in the short term.
An anecdote: my 18-month-old can't help but dance every time music comes on. She can happily step back and forth to the beat, knowing it will come every time. If the beat suddenly doesn't come... she will sometimes throw a fit.
But it becomes more complicated when considering lyrics, and sounds typical in your culture. An older blues musicians once told me there's only two beats in blues: the horse-gallop and the train-chug: two rhythmic sounds that were typical in early America (where blues was born). I notice that Celtic music has the constant thump thump thump, like an armorer hammering an anvil.
Of course, this is all speculative, and while I think the neuroscience is interesting, I think it has a lot more to do with psychology and sociology than neuroscience since it's such an emergent phenomena.