LagrangeEuler said:
Is the ground state always most probable state of the system? For example in problem of LHO or potential well?
In a closed system where the Hamiltonian is time-independent, the system will stay in the same state until it is otherwise perturbed. This is the case for the examples that you provide. Given energy eigenstates |i\rangle with energy eigenvalues E_i, we assume that the system is in the state |\psi(0)\rangle at time t=0:
|\psi(0)\rangle = \sum_i c_i |i\rangle.
At a later time, the system will be in the state
|\psi(t)\rangle = \sum_i c_i e^{-iE_it/\hbar} |i\rangle.
The time dependence of the state is completely encoded in phase factors.
The probability that a measurement of the energy finds a particular value E_i (so that we would conclude the system is in the state |i\rangle, ignoring degeneracies) is
P_i(t) = | c_i|^2,
which is independent of time.
If the Hamiltonian has a time-dependent part, it can be possible to have transitions between states. Suppose that we add a small perturbation V(t) to the system above. Then we can show that, to first order in the perturbation, the state at time t will be
|\psi(t)\rangle = \sum_i c_i (t) |i\rangle,
c_i(t) = c_i - \frac{i}{\hbar} \sum_j \int_0^t dt' \langle i | V(t')|j\rangle c_j e^{-i(E_j-E_i)/\hbar}.
Whether or not the probability to find the system in the ground state is an increasing or decreasing function depends on the type of perturbation. For instance, the perturbation could be a driving term, where we are pumping energy into the system from the outside. Then the probability to find the system in the ground state would actually decrease with time.
If the system is allowed to remove energy, perhaps by emitting photons, then the system will tend to settle into the ground state if there is nothing else to forbid it.