It's easier to look at the base units than derived units, IMO - there are less of them. Si base units are a good starting point as jcsd mentioned.
Wikipedia lists them as
meter, kilogram, second, ampere, kelvin, mole, candela
You can start paring base units down from there.
If you go down to Planck units, you can get rid of _all_ the base units. This may be going too far, I would suggest keeping the second as a base unit. A succinct way of saying this is that I'm a fan of geometric units (which shouldn't be a huge surprise to anyone who has read a lot of my posts).
This keeps Plancks's constant as an experimentally measured constant. The meter is already defined in terms of the second, so it's not really fundamental anymore. Defining mass in terms of seconds requires good values for c (which we have, it's already a defined quantity rather than a measured one in SI), and G (which we don't have which is why it isn't done already). Setting G=c=1 defines the geometric units for the most part (except for charge, which we get to next).
The charge of the electron would be sufficient to get rid of the ampere as a fundamental unit. There's another route to getting rid of the ampere/coulomb as a fundamental unit as well, that's to set the electric permitivity of free space to one - that's the approach used in geometric units. Either way, charge can be eliminated as a unit, and an ampere is just a columb/second. Kelvin is just another name for energy. Moles are just a convenient number. The candela is (I think), an artifact of the human visual system as opposed to being a fundamental physical unit. (I don't use candela's much, I'm fairly sure that's what they are there for). That takes care of all of the SI base units :-)
So we are left with one unit, the second, to represent (as Weyl says), the scale dependence of the universe - or, we can get rid of that by making hbar =1, and then not have any units at all.
The downside of not having units is the same as the downside of not having "types" in a programming language - it's a lot easier to make errors in an untyped programming language, it's a lot easier to make errors in a unitless physical system.