The other possibility is that while there may potentially be large extra dimensions, we are confined onto a surface in this higher-dimensional space. To take Nabeshin's sheet of paper example, the sheet of paper is sort of a two-dimensional object in three-dimensional space. But imagine that that sheet of paper is now made into a perfect two-dimensional object, with little two-dimensional beings in it. They can't interact with the larger three-dimensional world, and so can't ever observe it.
One thing that distinguishes this view from the one Nabeshin pointed out, of the extra dimensions simply being small, is that while it is easy to confine the forces like electricity and magnetism or the nuclear forces to such a surface, gravity cannot be contained to the surface. This has interesting implications in that it may potentially explain why gravity is so weak compared to the other forces: it is spread over a much large volume.
If the extra dimensions are small, then all forces are confined in the exact same way. Note that reality might be a combination of the two approaches: we could have, say, two large extra dimensions and a bunch of small extra dimensions.