stevil said:
Is SpaceTime infinite in all directions?
That depends on the particular spacetime. Some are, some aren't.
stevil said:
I thought in a closed universe it's kind of like a bubble.
A closed FRW universe is spatially finite (but without boundary, as you note--topologically it's a 3-sphere), and also finite in time--it has an initial singularity, and (at least in the case of zero cosmological constant) a final singularity as well.
stevil said:
there is no outside of the universe
Yes. You can try to visualize the universe as embedded in some higher dimensional Euclidean space (as you do), but that causes more problems than it solves.
stevil said:
I was wondering if the inside of a black hole were like that too.
Like what? If you mean, like a closed FRW universe, no, it isn't. There are spacelike surfaces inside a black hole that are spatially infinite. As for trying to visualize an embedding of a black hole interior in a higher dimensional space, that's even more problematic than doing it for a closed FRW universe.
stevil said:
The alternative is singularities
Singularities aren't an "alternative" to anything. They are present in some spacetimes (such as both of the examples given above, a closed FRW universe and a black hole interior) but not in others. It just depends on the particular spacetime.
stevil said:
Singularities seem like nonsense to me
Many physicists believe that the presence of singularities in particular classical spacetime models is a sign that those models break down in those regimes, and that some new theory, such as quantum gravity, will be needed to model what actually happens in regimes where our current models have singularities. However, that is still an open area of research.