It's important to keep in mind that the Earth would be a cold, cold place if there weren't any greenhouse gases in the atmosphere. It is those greenhouse gases that prevent the atmosphere from achieving a thermodynamic equilibrium, with a more or less constant temperature profile from the near the surface to the top of the atmosphere.
The atmosphere instead exhibits a temperature that decreases with altitude, a drop of about 6.4°C/km on average. This is the atmospheric lapse rate (google that term). While this lapse rate is driven by adiabatic cooling, this nonetheless goes against the grain of the second law of thermodynamics. Something has to be responsible driving the atmosphere to this non-equilibrium condition. That something is a combination of the heating by the Sun and the presence of greenhouse gases in the atmosphere.
The predominant components of the atmosphere are diatomic oxygen and nitrogen. Being diatomic gases, these are more or less transparent to infrared. The atmosphere would need to be much thicker to get a greenhouse-type effect from oxygen and nitrogen. On the other hand, multiatomic gases such as water vapor, carbon dioxide, and methane are rather opaque to infrared, even in dilute quantities. It is this opacity to infrared that drives the atmosphere toward having something close to an adiabatic lapse rate as opposed to a constant temperature from near the bottom to near the top. Increasing the quantity of these multiatomic gases changes the temperature profile of the atmosphere.