I Data collected from different devices: how to combine for analysis?

  • I
  • Thread starter Thread starter Mikki123
  • Start date Start date
  • Tags Tags
    Data
AI Thread Summary
Combining data from different devices for analysis requires ensuring that the data is comparable, which may involve selecting a common physical quantity to represent. The x-axis should ideally represent time, and the data should be collected at regular intervals to avoid inconsistencies. Fourier transformation may help in identifying interferences and making the datasets more comparable, but understanding the measurement context is crucial. Proper experimental design and knowledge of the instrumentation are essential for meaningful data analysis. Without this foundational understanding, the data may lack significance, leading to poor performance in machine learning models.
Mikki123
Messages
4
Reaction score
3
Hi Everyone,
I'm working on a project where I have current values from three different devices when there is no arc and an arc generated by an arc generator. When I plot them, they all look different since the data is from different devices. Is there anything I can do to make them comparable, like make them look similar, so that I can perform further analysis?
 
Physics news on Phys.org
In your plot where y is current values, what is x ?
 
it is just indexes starting from 0 to the number of samples
 
However you may get statistics, e.g. average, sandard deviation, as mathematical treatment, Number has no physical meaning. You had better pick up some phisical quantitiy from the samples for plot,e.g. same divice with different physical condition, same condition with different devices.
 
  • Like
  • Informative
Likes Vanadium 50, russ_watters and mcastillo356
All good. I will try doing that. Thankyou :smile:
 
Mikki123 said:
it is just indexes starting from 0 to the number of samples
Are they collected over time? Then x is time, isn't it? Are these samples collected in regular periods? Then it is just a matter of knowing frequency, no?
 
  • Like
Likes russ_watters
Hi Borek,
I just have a single column of 800,000 current values. The x values should be time, I suppose. I have the same from three different devices. But while plotting it, they looked so different. I wanted to train my machine learning model with this data for further processing. Since the data all looks so different, I'm getting such poor performance. Do you think the Fourier transformation for all three will make them look similar so that I can better train my model. I'm looking for any kind of preprocessing apart from normalization and feature extraction
 
Mikki123 said:
The x values should be time, I suppose.
You should probably know this (?????) It matters that they be equally spaced with no " jitter ".
Then look at (the difference) fourier transform to find interferences.
When you say feature extraction what exactly do you mean?
Why do you expect similar results?
 
  • Like
Likes russ_watters
Good data requires a detailed understanding of what it really means; how the measurement instrument works; what it is REALLY measuring. Good lab work is more about experiment design and selecting and/or researching the instrumentation than it is about collecting the data. This would be especially true if you are trying to represent the same physical quantity with different methods.

Without this prior engineering it is likely that the data is meaningless, or has unknown meaning. Bad data can be combined however you like. Garbage in, garbage out applies from the very beginning of experimentation and analysis.

A set of numbers isn't data, it's just numbers. Data has an associated meaning and context.

You will not get useful answers from us if we don't know, in detail, what you are measuring, why, and how.
 
  • Like
Likes gleem, Vanadium 50, hutchphd and 2 others
Back
Top