I'm aware that it helps keep warm things warm—but does it similarly help keep cool things cool?
Sure. But rather than thinking in terms of keeping stuff cool or warm, think of it as reducing heat transfer.
It reflects thermal radiation, so it acts as radiative insulation.
Emergency first aid-type blankets are meant to prevent loses of body warmth. The heat transfer from the body to the colder environment is reduced.
In cryogenic applications the so-called "superinsulation" is just aluminium foil wrapped around a vessel. Here, the environment is warmer than the cryogenic liquid inside the vessel, and the foil acts to reduce heat transfer inwards.
Aluminum foil also looks cool, especially if you wear it back-to-front.
Separate names with a comma.