That's a good question. I like to see it this way: The exponential function solves a fix point problem, namely ##y'=y##. A general differential equation is a kind of variation of it, ##y'=f(y)##, so it's plausible that a some sort of modification of the exponential function, ##c \cdot e^{g(x)}##, comes into play.
Another point of view is the fundamental (and defining!) formula ##e^x\cdot e^y=e^{x+y}\,.## It translates the multiplicative structure of some curved phase space, which the solutions to a differential equation system are, into the linearity of its tangent spaces, or for short ##e^0=1## or ##e^{\operatorname{trace}}=\det##. In a differential equation system we have some statements about tangent spaces, i.e. something linear: ##x+y##, and we are searching for the curved solution to which these are the tangent spaces, i.e. ##x+y \mapsto f(x)\cdot f(y)\,.## And such a relation defines the exponential function. The next prominent functions are periodic ones, the trigonometric functions. However, a little detour to the complex numbers shows us, that they are, too, a combination of exponential functions.
So for me it is the property ##e^{\text{linear}} = \text{curved}\; , \;e^{\text{tangents}}=\text{flow}\,.##