Well, I have no authority on this except that I'm an avid student of the history of empires, but I would get the Iberian (Spanish and Portuguese) and--believe it or not--the Belgians may win this one.
The two Iberian powers ruled over most of the Americas, and did so with a brutality probably worse than the French, British, or Americans. They used slave labor extensively, and even more cruelly, than the Anglo-American powers did-- see the Potosi Mine for one example of this (
http://en.wikipedia.org/wiki/Potos%C3%AD), or the extensive sugar plantations throughout the Caribbean.
The Belgians are notorious mostly for one king, and one place: King Leopold II's Congo. It was the King's private property, more or less, and he raped it so horribly that the Belgian Congress had to wrest control from him eventually. Altogether, it is estimated that AT LEAST 10 million people died under his reign from being worked to death or massacred--not to mention some of its possible echoes in things like the Rwandan genocide. This is an excellent book on the subject:
http://en.wikipedia.org/wiki/King_Leopold%27s_Ghost and here's a further link on the Belgian congo:
http://en.wikipedia.org/wiki/Congo_Free_State
Hope this helps. Don't forget the British, either. They weren't the cruelest, but they were by far the largest empire, which adds up.
