Third argument
"The universe is a closed system. The second law of thermodynamics describe that the amount of usueable energy must decrease. Since there are still amounts of usueable energy, this means that time must have a beginning"
Also this argument, like argment #1 has immense popularity.
It can be shown that in this case the laws of thermodynamics are taken out of context. And while the law which is used here to argument is called the second law of thermodynamics, ordinary logic will tell us that physics also came up with a first law of thermodynamics. It will be good to study this one first, and not hop immediately to the second law, after all the fact that they have been given numbers must indicate somehow that the first law takes precedence over and is more important then the second law.
So let's see what these laws state:
First law:
In a closed system the amount of energy and matter are conserved quantities.
Second law:
In a closed system, the amount of usueable energy must always decrease.
Critique
The universe is a closed system?
The definition of a closed system is one that is not in thermal contact with the surrounding environment. There is no interchange of energy, neither matter. It is however arguable that we can not extend the definition of a closed system, which in normal cases is only applied to finite systems, to that of the universe. When we also consider the first law, this means that if we consider the universe to be a closed system the amount of matter and energy must be conserved.
The combination of these two findings will result in an universe, in which the total amount of matter and energy remained constant throughout time, and since they are conserved, we can not think of a begin of time, while the amount of usueable energy must be decreasing. This obviously leads to the paradox then that the usueable amount of energy should have already been gone to zero.
We go outside, and see: the sun is still shining.
Something must be wrong then, cause our 'theory' would indicate, all amounts of usueable energy would have already been used!
a begin of time?
As a "way out" of this contradiction, it is concluded that "time must have a begin" then. But what good is this "solution"?
In fact, our very first statement was that the universe was a "closed system". Since the first law is still applicable, it must have been the case then that, since we assume that "time had a begin", this system all of a sudden had 'started' and became filled with matter and energy from appearently nowhere. A flagrant violation of the first law, since we hold it that  if we take it for our argument that the universe is closed in using the argument from the second law, it is not any less closed when we use the argument of the first law!
Solution
A closed system is in fact always denoting a finite system.
We could consequtively 'built' a system, by adding subsequently all the systems together that are in thermal contact with the initial open system we start with, until we reach a 'closed' system, a system that has no thermal contact with anything outside it. The point is however, that such is undoable, since we are always left with the rest of the universe, with which we are still in thermal contact, and thus did not never achieve our 'closed' system.
Conclusion
Our initial assumption that the universe is a 'closed' system, must therefore be obviously wrong.
