You could be way off, but if so, I don't notice it (yet): for your remark sounds insightful to me.
Early people recorded events by means of positions of Sun and moon, and thus the "time" concept emerged based on observing (and counting in one direction!) the motion of natural clocks.
Clocks actually are a measure of, as you already hinted at, the progress of natural processes. Thus we have the solar clock, (moon) months, water clocks and "radio clocks" such as C14 as well as "atomic clocks". However, people also have an intuition of a "flow of time" that doesn't exactly correspond to clocks; perhaps that difference in perception is what bugs people. In physics we can only deal with clocks.