If I were in your shoes, I wouldn't.
Data Science is an extremely employable major (and fewer schools have it than plain old math, even if they teach the material).
And, I've never heard of someone being rejected from a PhD program because one of their double majors was irrelevant to the program. I know people who were admitted to PhD programs in physics who majored in physics and in piano performance, for example (my undergraduate college had a pretty well regarded music conservatory, so this was pretty common there).
Indeed, big experiment HEP and astrophysics these days have entered the era of "big data" where the cutting edge science primarily involves thoughtful and accurate mining of data collected on an automated basis in amounts too vast to be understood raw by a human being.
Having experimental physics and astrophysics and non-physics data science work as Plans B, C, and D is a good position to be in. Admissions to top Theoretical Physics PhD programs is very competitive and finding employment in that field afterwards is too.
The rule of thumb used to be (and maybe I'm an old fart who is out of date on this) that there are 100 applicants for every theoretical physics job post. An uncomfortably large share of people who earn PhDs in theoretical physics ultimately abandon that career, sometimes after earning a string of post-doc and assistant professor posts but failing to find a permanent position (Sabine Hossenfelder has written several high quality up to date, data based posts on the state of the theoretical physics job market but
this one is the only one I found quickly, and
Peter Woit has some commentary on a video she made on the subject).
4gravitons is another blog that has quite a few posts on the process of establishing a career in theoretical physics.
You could be the exception. But having the credentials necessary to give you a back up plan is a good idea.
Lots of physics majors ultimately end up in IT. For example, my sister-in-law was an undergraduate physics major, and was planning on pursuing graduate work in it eventually, but the demand for the side knowledge of IT she gained in the process was so great that she ended up doing that right out of college instead, and in the end, she never went back to physics professionally. A physicist with a relevant credential has a leg up in making a transition to a Plan B in IT if that becomes necessary (and given the tight IT market after the latest round of mass layoffs in big IT firms, you might need a leg up).
Also, consider that you could develop some sub-speciality, e.g., technicolor theory, or preon theory, that five or ten years into your professional life, could go bust when it becomes strongly disfavored by new experiments. This could leave you adrift professionally, starting over almost from scratch, even if you find a tenure track position, because it would make it difficult or impossible for you to meet the publish or perish expectations for you as a theoretical physicist to actually get tenure and keep your job. This is not a mere hypothetical possibility. It has happened to lots of physicists in my lifetime, and it has happened even in your, probably thirty years younger, lifetime.
If you want to strengthen your modern physics and advanced math knowledge, nothing prevents you from taking those courses within your current major (I would echo the usefulness in theoretical physics of abstract algebra/group theory). If this is your passion, sacrifice elective courses in history or art and take some of those courses.
Take course in many different areas of physics, especially those you know least about, and figure out what you like.
There were certainly areas for me that I found superficially interesting when I was an undergraduate, but decided not to pursue further once I found out what the current dominant research programs in those fields actually involved (e.g. macroeconomics).
You may think that theoretical physics is the way to go now, but once you take a condensed matter physics course, which was a subfield you didn't know much about one way or the other, you might discover that this is really your thing.
Another good way to get a better sense of what fields you might want to specialize in is to
read pre-prints at arXiv on a regular basis in the fields that you are considering, to see what the active areas of investigation are in various fields, what methods they use, and what catches your fancy. As you do so, pay attention to the authors of papers you find insightful, who could be your potential advisors in a PhD program. Keep a list, you don't have to memorize them right now. Even if you don't end up working with those particular people, familiarity with the names of physicists who are publishing in the field and what the bleeding edge developments that they are working on impresses interviewers in the PhD application progress.
Reading these papers will also help you learn where the holes in your existing knowledge are, allowing you to patch some of those gaps with undergraduate course work before you apply to PhD programs. But, also, don't sweat it too much. Graduate programs are in the business of teaching you what you don't know and filling gaps in your knowledge. If you've covered the basics, there is no shame in having to learn some advanced topic for the first time in graduate school. This is literally what graduate school is for. If the only place you can learn what you want to learn is in a graduate school course, take it. But don't consider it a pre-requisite to admission to a graduate program.
Also, take advantage of being at a big, good quality university like Northeastern and being "a research assistant in this
lab and was a research assistant in another lab last summer."
You have a unique opportunity to rub shoulders with and talk casually with a decent sized pool of physicists who have made it, and they will be happy to spend time discussing things with a major in their department who has also shown commitment to the field by being a research assistant two times in a row.
Helping you at this point in your life is literally their job. Bounce ideas off of, and ask questions of, as many professors and graduate students in physics and math as you can.
It is a golden opportunity to gather intel and you don't even have to endure the awkward moments of introducing yourself to someone in a cold call to ask their questions. In the academic and career advising world they call these discussions "informational interviews" but you don't have to be so formal about it. Research labs have lots of down time waiting around for the experiments to do their things. Make small talk with the graduate students and professors running them about different future possibilities and what the state of research is in different fields.
Become clear about the differences between pure theoretical physics work and experimental work, and the places in between like statistical analysis and phenomenological calculations of Standard Model predictions at big HEP collaborations. Ask about what areas over oversubscribed and what areas don't get the attention that they deserve.
Discover niches that you weren't even aware existed. There is a good chance that your ultimate career path will be some sub-field that you've never even heard of yet.
Full disclosure: I'm a lawyer and physics is a long time hobby for me. I was a math major and about a course short of a physics major as an undergraduate (graduating in three years to save money). I was good at math and got decent grades in my major, but not PhD level good or PhD level driven to do research in math for the rest of my life. I considered going back to get a PhD later to the point of taking the GREs but life got in the way. But I lived all of my life through finishing law school and beyond, in and around academia. I started reading the Chronicle Of Higher Education on a regular basis when I was 12 or 13. My father was a STEM program director at a university and my mom was a university administrator (primarily she helped professors make their way through the grant application progress and dealt with applying ethics requirements to human subjects research proposals). Almost all of my parent's friends and my friends' parents were academics. My wife spent time as a graduate teaching assistant, as a lecturer at a university, and as a university admissions officer (primarily for international students). In college, as a student government representative, I served as a student representative on faculty-administration committees that hired professors and decided which departments should get new positions or lose existing ones. Many of my close friends became professors and we talk about it. So, while I didn't personally take this path, I'm very familiar with how graduate education and career paths in this area work.