The p and n regions are at different intrinsic potentials, with an energy (voltage) gap preventing current flow. The charge carriers can't flow "uphill." When you apply a reverse bias, the gap gets bigger and again no current flows (except for a little leakage). When you apply a forward bias voltage, it narrows the gap. At room temperature in Silicon, 0.6 to 0.7V sufficiently flattens the gap that carriers can flood across.
The exponential behavior comes from the thermal Boltzmann distribution of carrier energies. At room temperature some carriers have thermal (kinetic) energy, but energetic ones are exponentially less numerous than slow or stationary ones. Carriers with more energy than the gap can flow across (jump the hill), carrying a current. As you increase the forward bias voltage, exponentially more carriers have enough energy. That's why the I-V curve is exponential.