Close. If we take the square root:
E = sqrt((pc)^2 + (mc^2)^2).
Dirac was looking for a way to express the square root as an operator, and came up with the idea of using matrices to represent the operation. But there were negative solutions for E, for the same reason as you gave, but all without dropping the term (pc)^2.
Dirac assumed that the negative solutions were already all filled up with some infinite sea of negative energy electrons. The "holes" in the negative electron energy levels behaved like positive energy positively charged electrons, e.g. positrons.
Really, Dirac's reasoning here isn't rigorously correct, but he did get the right prediction. To me, it seems he accidentally got the right prediction with a pretty bogus explanation.