# Other linear fitting than least squares

Gold Member
I'm analysing some data and my task is to get a line that best fits the data, using least square I'm getting these dashed curves (red and blue) with low correlation factors. Is there another method that takes into consideration the amount of data placed into the direction of a line?

#### Attachments

• graph.png
31.6 KB · Views: 313

phyzguy
It seems like you must be doing something wrong with your least-squares fit. Can you calculate the least-squares value for the red, blue, and black lines? It looks lower for the black line.

FactChecker
Dale
Mentor
2020 Award
Those red and blue lines don’t look right. I think there must be something wrong in your code

Gold Member
the black line is just the identity y=x. The red and blue I got through my data.
I used the scipy.stats.linregress, can't see what's wrong but I'll take a look again

Gold Member
Oh there was some data hiding beyond my axis limits. Sorry for this mistake, I'll fix it and post the result.

#### Attachments

• graph2.png
19.1 KB · Views: 200
Dale
Mentor
2020 Award
So if you do a leverage plot that one datapoint will probably have a huge leverage. I would check that point and see if there is some error. Like maybe a typo when copying the data.

FactChecker
FactChecker
Gold Member
Aha! So the standard least squared regression is doing a good job on the entire data set. But you should consider that the data visible in your first post looks like it is following a different rule than the entire set. If you can see the reason for that, you may want to analyse the data in sections that make more sense.

Felipe Lincoln
Stephen Tashi
Why do you think fitting a straight line to that data (however it is done) would be a good idea?

Gold Member
Why do you think fitting a straight line to that data (however it is done) would be a good idea?
It is an experimental data that is expected to have a linear correspondence.
I just removed the zeroes data and this is what I got now. Thank you all

#### Attachments

• graph.png
27.6 KB · Views: 172
Dale
Mentor
2020 Award
I just removed the zeroes data and this is what I got now.
Do you have any experimental justification for that?

Gold Member
Do you have any experimental justification for that?
Yes sir. The zeroes was generate by my code to represent experiments that failed and resulted in no data.

Dale
Mentor
2020 Award
Yes sir. The zeroes was generate by my code to represent experiments that failed and resulted in no data.
That is an excellent reason!

It is never a good idea to throw away data just because it makes your fit better, but if a data point is bad for some specific reason then throwing it out is acceptable.

Felipe Lincoln
Dale
Mentor
2020 Award
FYI, you may want to also look into that data point with the high Rphenix. It looks like it has a very high leverage and it may have some other problem.

FactChecker
Gold Member
Right!! The next step in our research is to analyse what was the experiment conditions that bring some points a bit far from the expected. Thanks for your attention Dale!

Dale