If you are imagining the big bang as a some kind of bomb explosion, that's a really wrong description.
You are still thinking as a point explosion.
Okay let's think like this. You have a plane with an "infinite" size. This will be the description of the universe. It's inifite, so you can't think of any edge or something else.
You are a creature that living on that surface. Now try to think like this; infinite plane divided be equal size squares. Each square has an area of ##a^2##.
1-How many squares are they ?
The answer is infinite. Why ? Because universe is infinite
2-What's the total area of the universe ?
Well it's simple each square has an area of ##a^2## and there are infinite sqaures so the answer can be found by,
##a^2.∞## which its equal to ##∞##.
Now, let's think what happens in an expanding universe. The area of each square increases. Let's suppose it increased by twice respect to the current size.
3-Whats the area of the each square ?
Well simply ##2a^2##.
4-Whats the total area of the universe?
Again, each square has ##2a^2## area and there are infinite squares so the answer would be,
##2a^2.∞=∞##
Well the universe expanded twice but since there are infinite squares the universe is still infinite.
Now let's come to the case where that, we are all interested. "What happens when each size of these squares gets smaller and smaller ?""
Lets suppose the squares are shrinked to a size where its area is now ##(\frac {a} {4})^2=\frac {a^2} {16}##
5-What is the total area of the universe ?
Well each square has a size of ##\frac {a^2} {16}## and there are infinite squares, so the answer is still infinite.
The important thing is that we can do this process until a point where the area of the square reaches nearly zero. For example, area of the each square can be ##0.000000000000001a^2## but since there are "infinite" squares the total area of the universe will be again, infinite.
The cruical point is, the universe is still infinite at this "after big bang" stage. Universe is still infinite and you can't think this, like a growing thing explosion.
Now the important question is "What happens when the size of the each square becomes zero"
Let's try to calculate it. The area of each square is "0" but there are infinite squares, hence we get ##0.∞## which its "undefined".
This is the main problem that we are dealing with. Also this is the singularity that we can call.
You and every point on this "plane" universe can shrink to a state where the size gets till zero but at zero it becomes a singularity and our equations don't work.
Why we say "it happened everywhere" ?
As I said before each square on this infinite plane shrinks to an event that we call singularity.
Hope this helps