Find Trend in STAT: How Many Observations Needed?

  • Thread starter Thread starter knightza
  • Start date Start date
Click For Summary

Discussion Overview

The discussion revolves around determining the number of observations needed to establish the significance of a trend in statistical analysis, particularly in the context of analyzing a large dataset related to temperature, speed, and pressure. Participants explore concepts related to statistical significance, confidence levels, and the challenges of data analysis in a specific research project.

Discussion Character

  • Exploratory
  • Technical explanation
  • Homework-related
  • Debate/contested

Main Points Raised

  • One participant inquires about the number of observations required to prove a trend's significance.
  • Another participant suggests that significance is determined by confidence levels rather than the number of observations.
  • A participant mentions analyzing a dataset of around 7000 observations and seeks guidance on handling "strange observations" within a 95% confidence level.
  • One response introduces the concept of system identification and emphasizes the complexity of the analysis, suggesting that a trained individual should handle it.
  • Another participant notes that the clarity of trends in the data can influence the choice of statistical functions and the interpretation of R² values.
  • A participant expresses concern about their qualifications for the project and considers consulting their supervisor.
  • Further discussion raises questions about the time history of the data and the implications of errors in the dataset due to configuration issues.
  • A participant seeks clarification on whether to analyze data from individual tyres separately or consider interdependencies between them.

Areas of Agreement / Disagreement

Participants express differing views on how to approach the analysis of trends and the significance of observations. There is no consensus on the best method for determining significance or handling the dataset's complexities.

Contextual Notes

Participants acknowledge limitations in their statistical backgrounds and the potential for errors in the dataset, which may affect the analysis. The discussion highlights the need for a solid understanding of statistical methods and the specific context of the data being analyzed.

Who May Find This Useful

This discussion may be useful for students or researchers engaged in statistical analysis, particularly those dealing with large datasets and seeking to understand the implications of data quality and trend significance.

knightza
Messages
11
Reaction score
0
Hey, does anyone know how many observation is needed before we can prove that a trend is significance?
 
Engineering news on Phys.org
You don't prove a trend is significant based on the number of observed points, its based on the confidence level.
 
Hey Cyrus, thanks for your reply, sorry I'm not very good at STAT here but I currently have to analyse quite a huge set of data. The first lot they sent me has around 7000 observations (and I'm trying to relate the temperature with speed, pressure etc) so how many "strange observations" am I allowed if, say, i want it to be with in 95% confidence level. Is there some formula to calculate this??

Thanks

Knight
 
What you are trying to do is system identification. There is a large body of theory behind it, along with statistical metrics to determine what terms to include in your model for explanatory variables. If you do not know how to do this, I would recommend you give it to someone that does because this is not something you can teach yourself in short order and it requires someone trained in SYSID.

It doesn't work based on "strange observations," I think the problem is that you don't have a fundamental background in statistics. :frown:
 
OMG this is so..sad. why did they give this research project to a third year uni student T.T

Ok, thanks for your help anyways, aye? I might need to talk to my supervisor about it.
 
knightza said:
OMG this is so..sad. why did they give this research project to a third year uni student T.T

Ok, thanks for your help anyways, aye? I might need to talk to my supervisor about it.

It all depends on what your data looks like. If the data has very clear distinct trends, then you can probably get away with using some simple functions and seeing what the R^2 value looks like. If you have a more complicated system, then things will be nasty pretty quickly. Do you know what the time history of your signals look like?
 
What did you mean by time history though?

This is quite a specific project on a specific mine site and the data was just recorded every minute since 25th November.

I am now waiting for the second lot of data since it was found that the data on the first lot contains many errors in it (due to configurations, installation of the device and so on).

Hopefully the relationship will not be that complicated. I'll certainly get back to you once I have more problems (most likely when I get the second lot of data).

Thanks

Knight


PS

One thing that I need the clarification for, say, if we are looking at the data from the tyres of one vehicle. Should I be analysing the data of each tyre separately? both temperature, pressure etc. Or is there a possibility that the increase in pressure/temperature of one tyre will cause an increase in the other tyre too?
 

Similar threads

What Are the Top Engineering Topics That Are Currently Trending?
  • · Replies 4 ·
Replies
4
Views
3K
Undergrad Time series and why would we remove seasonality and trend?
  • · Replies 10 ·
Replies
10
Views
4K
High School Applying Bernoulli's equation to problems involving a perfect gas
  • · Replies 8 ·
Replies
8
Views
1K
Other Finding a research topic in the Aerospace field
  • · Replies 11 ·
Replies
11
Views
3K
Graduate How Many People Can Perform Quantum Mechanical Calculations?
  • · Replies 10 ·
Replies
10
Views
2K
High School How do Black Holes Grow? A Far-Away Observer's Perspective
  • · Replies 67 ·
3
Replies
67
Views
6K
Insights How AI Will Revolutionize STEM and Society
  • · Replies 77 ·
3
Replies
77
Views
7K
Lawn/Garden Determining Ramp Length for Ground Clearance
  • · Replies 8 ·
Replies
8
Views
1K
Undergrad Many worlds and interference
  • · Replies 4 ·
Replies
4
Views
3K
Undergrad How Does the Many-Worlds Interpretation Address Nonlocality?
  • · Replies 10 ·
Replies
10
Views
3K