You have not closed any doors yet (assuming your grades get you into a university or college with a physics program). However it may be that academic advising at your intended university or college may suggest you with your B's that you do not take the honors physics course (if there is one at your university). Almost always, you can still major in physics. The first course in physics is usually part of a sequence of courses. If you are not in honors physics, you may be able to get into the honors sequence for subsequent courses with good grades in the physics course(s) you are taking. At my undergrad school we had 4 different sequences: a four semester sequence for physics majors; a three semester sequence for engineers; a two semester sequence for life sciences; and a semester course for acquaintance. When I graduated as a physics major (about 12 of us), I would say 9 -10 were from the honors sequence and 2-3 were from the engineering sequence.
You may (or may not, I'm not sure (probably depends on the enrollment) ) be able to enroll in the physics sequence even against academic advising. However if you ever get on academic probation, and went against their advice given, this could be trouble (if they remember).
Any awful possiblility of probation could be at least 1-2 years away. For now, I do not think you have closed any doors in physics, yet. But...
Follow the good advice of others in this thread. Learn to work hard now in physics and math. Do as much as you can to remedy your problems in test-taking. I grant you this may be challenging. I did terrible on my first tests in college. After the first half dozen tests or so, you get used to taking tests and your grades usually improve.
By the way, my grades in HS were always best in Chemistry (all A's for two years) , and A-'s in Physics. I always found HS Chemistry more interesting. Yet I majored and got advanced degrees in Physics.