Zandorian: do you enjoy games; if so then perhaps you can gear that towards a passion in creating Physics simulations OR realtimes Physics Simulations. This way you would be integrating your studies of computer science with your interest in physics...and slowly you will begin your pursuit of a higher level in physics(with or without a academic foundation in physics)...
The key is to get started on coding your own 3D engine which will lead to a physics engine and then the higher more complex stuff.
I hope your college is geared towards programming...I suggest the following
[0] Look at libsdl.org (or search SDL-simpledirectmedia layer
[1] If you can get your hands on David Eberly's Code...Its not open source
so if you can't find it go buy the book because the majority comes with it.
[2] Pick up an OpenGL book (the bible, primatech series or redbook)
[3] gamedev.net
[4] ogre3d...u can start with ogre3d if you want because its prebuilt and you can begin coding physics stuff...but you'd need to learn to compile all the stuff.
[5] Pick up a good standard intro to physics text like Serway
[6] Numerical Recipes in C or C++(very IMPORTANT)
[7] Data Structures TExt(or look at STL)
[8] Flocks & boids may be a good first sim to do.
if you begin to build an interest in building sims that may lead you to a higher interest in building physics sims eg.
RigidBody,Particle COllisions,Billiards,Astrophys Sims(cosmo/stellar), thermal models, even moleculaar modelling, ragdoll physics, solid states/crystal geometry etc.
I found that once th ecoding interest and behavoiur to code non-stop was there the physics came naturally(but then again I did the physics first).
Once you have a proper rendering schematic...look to
code the
[0] first the Motion Equations,without mass
[1] motion equations with mass/Forces
[2] MultiBody Problems(learn scene Managment)
[3] Rotational Physics(i'm on this stage)
[4] Lastly whatever advanced physics you would like to go to...
(i'm looking to go into astrophys/planetary/molecular modelling
Its a long journey but i think its worth it.
Hope that helps.