There are a lot of myths about the “Big Bang” — the theoretical rapid expansion of space giving rise to the universe as we know it. One of the biggest ones is that the universe expanded from a point, and thus has some edge. Many of us picture the Big Bang as an explosion, throwing matter outward from some point in space. But really, there was no “center” to the expansion of the universe, and matter didn’t get thrown out from some particular spot. The observable part of the universe was packed into a very tiny volume, yes, but that volume wasn’t surrounded by empty space. If the universe is infinite today, then it was infinite at the time of the big bang as well. So outside that tiny volume was more matter and energy — it just isn’t observable to us. So, the early universe was extremely dense, but not necessarily extremely small.
Note, too, that the Big Bang isn’t the beginning of the universe. I specifically said that it’s the beginning of the universe as we know it. Before the Big Bang, the physical laws of the universe as we know them didn’t hold — no magnetism, gravity, none of that. Things were such a dense hot soup that we have very little idea of how things behaved at that time. The Big Bang assumes that space, time and energy already existed, but doesn’t tell us how the universe came to be dense and hot at that moment.
We know that space is expanding because distant galaxies are moving away from us. But they’re not just moving away from us (we’re not the center of the universe, much as we’d like to be), they’re moving away from every other galaxy. That said, they’re not actually moving… they’re just sitting there. Space itself is expanding, and as the space between galaxies expands it carries the galaxies further apart, like raisins in an expanding loaf of bread.
This was taken primarily from the “Brief Answers to Cosmic Questions” produced at the Harvard-Smithsonian Center for Astrophysics. Check it out, they have some great stuff!