This is how I would put it:
Entropy is a thermodynamical concept and can only be applied to a system that has a huge number of coupled degrees of freedom. In the idealized situation of a ball rolling down a hill one can't say that the entropy increase is what drives the ball downhill towards smaller potential energy. It's the Newton's or Lagrange's equations that drive the ball. The only mechanical degrees of freedom in that situation are the coordinates of the ball. In the real world, though, the potential energy of the ball rolling downhill is lost to frictional heating due to the motion, which excites the huge number of vibrational modes of the molecules on the surface of the hill, and entropy really increases.
In the case of a decaying excited atom, the combined system of the atom plus the surrounding electromagnetic field has very many degrees of freedom because EM field has an infinite number of normal modes (Fourier components). In that case one can argue that the atom decays because of the entropy increase that results.
Let's say we have a system of two coupled oscillators, initially at rest. Then at some moment we give oscillator A a 'kick' of kinetic energy. Because of the coupling, energy is tranferred to the oscillator B too, but the system still periodically returns to a state where oscillator A has all the kinetic plus potential energy. This is called 'Poincare recurrence'.
On the other hand, if we have a system of, like, a billion coupled oscillators, and we kick one of the oscillators, we will notice that energy will eventually be distributed pretty evenly to all of the oscillators, and the system never seems to move back towards a state where a minority of the oscillators have most of the total energy. The tendence of entropy to increase is what causes this. The Poincare recurrence time of the system of billion oscillators is so large that it's essentially infinite.