Using incorrect models to infer correct ones.

[FallenApple]

I see this happening a lot and have done this in my analysis as well. But it doesn't seem too intuitive to me as to why it works.

So if we fit a linear regression, and the residual vs fitted plots are curved, we tweak the input variables to get a random residual vs fitted plot. But we have used the wrong model to infer this to begin with.

For count data, if we use poisson regression and realize that there is over dispersion, we would then switch over to the negative binomial model. Here we have also used the wrong model to infer this.

I guess the reason why the incorrect models still give some insight even though they are not the best models is because they are somewhat in the right direction? Is that the case? Like even if a poisson model isn't fully appropriate, it is still somewhat of a decent approximation compared to other models, so we can at least say that there is some value in the results. All models are wrong, but some are useful.

 

[FactChecker]

Approximations or simplifications. If they tended to work in the sample data, we hope they will continue to work for new data points. If they worked very well, we hope that will continue.

 

[Dale]

All models are wrong, but some are useful.

That is along the lines of my thinking. It is similar to a Taylor series expansion, the more detail you need the more terms you use. No expansion is “right”, but it can be useful anyway.