Peter (IMC) said:
Would a Universe without any matter in it, be a singularity?
we have no evidence that spacetime singularities exist, except in the context of certain theories
by definition a singularity is not something in nature, it is a place where some theory breaks down, fails to compute meaningful numbers
historically the way that has be used to eliminate singularities is to improve the theory
several proposed replacements for vintage 1915 GR-based cosmology DO in fact remove the big bang singularity and extend further back in time---this is work in progress
So since as a general rule a singularity is not a physical reality, in any known case, it doesn't make sense to ask the question would a Universe without matter be a singularity. Would ANYTHING really ever be a singularity? I wouldn't know. Since they appear not to occur in nature, probably not.
============================
But there is an easy answer to your question that has nothing to do with singularities.
The de Sitter is an example of a universe with no matter in it. It has a positive cosmological constant. No ordinary matter or dark matter. It has no singularities. (no points at which the model blows up) Just a smooth bounce. Like a big balloon that gradually gets smaller until it reaches some pre-arranged size and then starts expanding again. The minimal size it reaches doesn't have to be especailly small---can still be billions of lighyears in circumference---depends on what parameters you put in the model.
de Sitter is considered a good approximation to our own universe in late times, when matter will have thinned out a lot due to expansion, so the effect of matter will be small and our own universe will behave very much like de Sitter.
the de Sitter model universe was discovered by a dutchman around 1917, as I recall.
there was just a great Scientific American article about it
see "my favorite SciAm article" link in the signature.