I will tell you about my experience, and it may relate to you or it may not. I graduated from the University with a very low GPA. I always had an interest in physics and I worked hard (usually I had to get stung by one or even two examinations in each course to begin to study in earnest). By the end of the course or the summer after, I had repaired all the lessons I should have learned the semester before. Finally in the last semester, I actually had a B+ average, (only for that semester). I had a hard time picking any professor to ask for a recommendation, but I knew I wanted to learn more physics in graduate school. I also knew from the last semester, that I could do the work.
I took the PGRE (without studying. This was > 35 years ago and I do not think you could study for it. I do not think there were any study guides back then and maybe ETS released one prior PGRE exam, I do not remember.) I got 58 percentile.I felt this was better than average in this regard. Anyway my confidence was shot. I did not apply to any graduate schools that year. (I think this was a mistake).
Anyway, I knew I would have trouble getting recommendations, but I really wanted to learn more physics. The business that I applied at the year before for a summer job had an opening (it was not related to academics at all but it was honest work and paid well), so I worked the year. Because I had money, I approached my graduate admissions at my university and asked can I enroll in Mathematical Methods in Physics (the graduate course). Graduate admissions told me to pay the tuition (back then it was only 440 (1978) dollars) and I would take the course for credit as a "special student". I get no benefits, no health care etc. like a real grad student would get. I figured this is fair.
I talked to the instructor (before going to graduate admissions) and he said this is OK with him. I asked him if I did well would he give me a recommendation. He said OK.
I think I got a B+ in the course while working > 40 hours a week. The prof wrote one of three recommendations, I had two others.
I applied to three schools the following year, none were in the top 20 in the rankings and at least one would be considered a safety school. I was admitted in all three.
Once I got into graduate schools there were a few rough places but nothing in comparison to what it took to get in, I definitely did well in the qualifying exams.
(this is a long story, and it may not be relevant as it was > 35 years ago. I am not sure I would take advice in 1978 from someone who went to grad school in 1940!)
The points I want to make is:
1. Improving GRE is laudable. You may want/need to follow it up with a course (if you can afford it and you think you can get a good grade). Getting recommendations when you have been out a while may be problematic. Also the grad committee may look favorably on a continued commitment to your physics study. Taking a course with your own money demonstrates you are willing to invest in yourself. Demonstrating that value would impress me if I were on the graduate committee.
2. At times, I took a couple of summers off to enjoy life and re-charging, back in the 80's it was called "finding myself". It never made a difference. You may not be more assured to commit the next 6 years after taking time off.
3. It may be just an impression, but I did also read online a power point presentation by a faculty member (I do not remember the University) the PGRE will help but will not save a poor GRE. A good GRE is more likely to save a poor PGRE.This would definitely apply in my case and it may not be applicable to you.