How much more would it take to get a double major?
Where do you train, poop-loops? I work 24 hours a week, am an EE student, plus I train about 3 nights a week(BJJ and MMA). It is doable.
Don't ditch your training. If you plan to compete, though, it will be tough.