Two effects. Main one is the fact that the fork is angled, so the front wheel turns in the direction you lean in. That makes you go along a curve, and that causes centrifugal force to push you straight again.
Second one is gyroscopic effect on the wheel. This one is significantly stronger on a motorcycle, but it contributes to the bicycle's stability as well.