Q-reeus said:
A rough way of looking at contributions of ρ vs p to central pressure is to note that the maximum relative gravitational mass contribution for p is when ρ = 3p (photon gas) - i.e. a 50:50 split.
You're correct that this is a limit on a physically possible solution for a fluid, but that doesn't by itself guarantee that the p <= 1/3 rho requirement can be satisfied for an arbitrary mass. In fact, the existence of this limit makes the constraints on a solution tighter, because now the pressure at the center doesn't just have to remain finite for a valid stable equilibrium; it has to also be less than 1/3 of rho at the center. You're correct that the physical meaning of that constraint is that an apparent "solution" that computes a pressure exceeding 1/3 rho will not be a valid static equilibrium; it will be unstable against collapse.
If you look at the relativistic hydrostatic equilibrium equation I wrote down, in the form of the Newtonian value times multiplicative factors, the limit of p <= 1/3 rho constrains the first factor (it will max out at 4/3 rho if p remains within the limit), but not the second; if m(r) -> 0 faster than r^3 p as r -> 0, the second factor will diverge (as in, really diverge--go to infinity). But even if it doesn't quite diverge, it can still cause p to exceed 1/3 rho, making the solution unstable against collapse.
Of course the obvious next question for you to ask is: how does the *Newtonian* equation of hydrostatic equilibrium behave when the p <= 1/3 rho constraint is imposed? Does it now predict some "solutions" that exceed that constraint and are therefore unstable? Off the top of my head, I would have to answer "probably yes", simply because the "standard" Newtonian analysis, which does not take into account *any* relativistic limits (essentially, ignoring the p <= 1/3 rho limit allows the speed of sound in the material to exceed the speed of light--under strict Newtonian theory it can go to infinity), has to allow the pressure gradient to become arbitrarily high in order to maintain static equilibrium as the total mass of the star goes up.
However, if I were you I would not go breaking open any bottles of champagne just yet, because I would also say, off the top of my head, that imposing the p <= 1/3 rho constraint but still trying to use a Newtonian equation of hydrostatic equilibrium is not consistent. If you are really interested in developing your own alternative theory of gravity that doesn't require pressure to be a source

, I would instead try the following "hydrostatic equilibrium" equation:
\frac{dp}{dr} = - \frac{\rho m}{r^{2}} \left( 1 - \frac{2m}{r} \right)^{-1}
As you can see, what I've done is taken the relativistic TOV equation and removed the pressure terms. The key is that the last multiplicative factor is left in; the physical argument for that is that it corrects for the fact that the radial coordinate r does not measure actual radial distance, so it at least respects the spacetime curvature aspects of relativity (but see below for a further caveat in the "extra credit" section

). If I'm right that the strict Newtonian equation will lead to *some* maximum mass, as I argued above, then this one should too, since the RHS is multiplied by a factor that is always at least 1 (if it were to become less than 1 the star would have collapsed to a black hole). [Edit: see correction in later post below.]
But the key question, of course, is *what* maximum mass? Can we actually obtain a neutron star mass limit similar to the standard one using this equation? Or does it lead to a limit that is way different (assuming it leads to a limit at all--I haven't proved that, just guessed it, and it's quite possible my off the cuff guess is wrong)? If you weren't allergic to math, this would be a great problem for you to tackle.

(For extra credit, you could also show how to derive the above equation for hydrostatic equilibrium from the "field equations" of your alternative theory. This would be a key requirement in any case to make an argument for the theory's consistency, since you would need some sort of field equation to derive equations of motion, conservation of the source, etc. and show that everything fits together. For example, I suspect, off the top of my head, that the alternate hydrostatic equilibrium equation above violates local energy conservation. If I were going to challenge your alternative theory of gravity, that's probably where I would start.

)
A final note: I fully expect you to say that the above is some kind of "admission" from me that since there *might* possibly be some alternate model that gives a mass limit for neutron stars without using pressure as a source, GR is somehow wrong, or inconsistent, or at least not proven. First, please bear in mind that I was careful to characterize what I said above as off the cuff, off the top of my head, etc. Second, please bear in mind that the standard GR model does not include pressure as a source just because physicists feel like it; the presence of the pressure terms in the standard TOV equation (which I gratuitously took out above) is *required* by the EFE and by the conservation law obeyed by the SET (covariant divergence = 0). Including the pressure is not optional in standard GR.
So what I did above is *not* legitimate physics; it's just hand-waving, of the same sort that I have complained about you doing. What I described in my "homework assignment" is some of the work that would have to be done to move the "model" I wrote down above at least some way in the direction of legitimate physics. But only some; there are whole piles of other data out there that I didn't even bother thinking about when I wrote down the off the cuff equation above. Is there a model that generates that equation and is also consistent with all of the other data? I have no idea. My personal judgment is that the off the cuff equation I wrote down is wrong: the correct equation of hydrostatic equilibrium is the standard GR one. Your judgment may differ, but judgment alone doesn't make either of us right or wrong.