Well, university is generally a level up from high school. The material is more advanced. The professors can be less forgiving than teachers. And in many places they're hired for their research skills with varying emphasis on teaching and in most cases have had far less training in how to teach. On top of that, if you're looking into physics or engineering programs, your peers will largely be made up of people who did well in physics and mathematics in high school and who have at least some kind of passion for these subjects. So you will have to bring your A game - so to speak.
Of course all of this can be intimidating - particularly if you haven't been leading the pack in high school. But it doesn't have to be.
If possible, try to adopt a positive mindset. You have an opportunity to study something that you love - most people in the world don't get this opportunity. You have the opportunity to be surrounded by other like-minded people, many of whom will share your passions and dreams and be happy that you share theirs. You'll have the opportunity to be taught be people who are doing cutting edge research in their fields, to ask them questions, and maybe even get involved with some projects they are working on. Sure, it will be a lot of work, but just about everything in life worth earning requires a lot of work.
You can also try to adopt a hard-working mindset. Some people believe that intelligence is innate - that you're born with a certain capacity that can't be increased. People with this mindset tend to do well until they are really challenged and then often fall victim to the believe that they have reached their innate limits. Other adopt a mindset that sees intelligence as more of a muscle - the more you work out, the stronger it gets. These are the ones that tend to do better in the long term, in my experience.
Finally, remember that you can always back down if you find you've bitten off too much. Balancing a full course load, a part-time job, extra-curricular commitments, and taking good care of yourself can be tough. If you really find it overwhelming there are things you can do to mitigate the stresses: drop classes, reduce hours at your job, drop an activity, etc. They all come with consequences, of course, but don't feel as though you have no options. Try to take good care of yourself though: get enough sleep, eat properly, exercise, socialise, etc. This will help you deal with any of the stress that you face in the long term.