It would be difficult to give each important course the attention it deserves if you spread yourself too thin. You could probably do it comfortably in 5 years.
Well, maybe if you take full advantage of summer school you can do it in 4 years comfortably.
If you plan to go to grad school, it would be a good idea to participate in research as an undergrad if you have the opportunity, but this might also consume a lot of time. And it's important to get good grades.
My first two quarters I took heavy course loads and regret it. This quarter I decided to just take 2 courses along with a research position which counts as elective credit, and I am much happier, and able to really learn the material in great depth.
When taking too many classes, I was overwhelmed with tedious homework and awkward schedules. I was just doing the bare minimum to get good grades, and I didn't get much out of it.
Also, at my University, difficulties can arise in getting a dual major, or even minor, due to scheduling conflicts. I am majoring in ECS and minoring in math, but so far almost all of the upper division math courses I plan to take have conflicted with CS classes I need to take.