I wish to know why and how does hysteresis occur from a purely mathematical perspective, using analytical functions and multivariable calculus. Thank you!
Since some phenomena and materials experience this and some don't, a "purely mathematical perspective" is impossible. Some physics will need to enter the problem.
From a purely mathematical point of view, hysteresis results when there is a cubic (or higher degree odd) dependence. If, in the phase plane (y versus dy/dt) y has a cubic dependence on dy/dt and we have the central part (in red in the graph below) unstable, then as x approaches 2 from below, it jumps to the other blue branch, going from 2 up. But as we come back down, x will stay on that section until it gets down to x= -2 before it jumps to the branch. (The x-axis is vertical in this graph.)
Hysteresis refers to an number of unrelated physical phenomena (and to some related ones) in mechanical systems, digital electronics, elasticity, electromagnetism to name but a few.
So if you would like to tell us which area you referring to please?