Register to reply

Data signal filtering

by DanatAMFL
Tags: data, filtering, signal
Share this thread:
DanatAMFL
#1
Dec19-12, 04:05 PM
P: 7
Hello all.

I have a data signal that I'm looking to reduce the noise in. I'm recording voltage at 12000hz and am getting high frequency noise induced by nearby power supplies in the lines that I am using. The noise amplitude is approximately .2-.3 Volts and the signal I am trying to get out has an amplitude of approximately .8-.9 volts. The baseline hovers around 4V (so when a sample is detected at the sensor, it jumps up to ~5v, then back down to 4V when the sample leaves the sensor, samples only last at the sensor for a fraction of a second). Does anyone know a filtering technique that can smooth the noise in order for my signal squaring program to square the sample data and neglect the noise? The squaring program works off a slope criterion; when it sees a steep slope for x number of points in a row, it raises the squared signal, and does the same thing for lowering it. I have tried using a median filter, however the noise amplitude seems to be too large for it to work effectively. If I increase the N value (number of points to sample) in the median filter too much, it greatly affects my measurement. I need the processed signal to climb as soon as the raw signal begins to climb for a legitimate sample. Ultimately I'm looking to integrate the signal in order to find a ratio between the total amount of time that the sensor is detecting a sample to the total time of measurement.

Thanks for any help you guys can suggest!
Phys.Org News Partner Engineering news on Phys.org
UConn makes 3-D copies of antique instrument parts
Amazon worker piloted drone around Space Needle
Five next-generation technologies for positioning, navigation and timing
yungman
#2
Dec19-12, 04:27 PM
P: 3,883
I would work on the analog front end to reduce the noise first. You need to provide the schematic, a few picture on the front end, the cable that connect from the detector to the amplifier. Power supply noise is not the hardest to get rid of, just provide more info.
carlgrace
#3
Dec19-12, 06:12 PM
P: 555
I agree with yungman, kill that noise at the source!

Make sure your power supply is properly bypassed... you shouldn't be getting HF from a power supply.

jim hardy
#4
Dec19-12, 07:30 PM
Sci Advisor
PF Gold
jim hardy's Avatar
P: 3,515
Data signal filtering

I'm ...... getting high frequency noise induced by nearby power supplies in the lines that I am using.
Switching power supplies are notorious for clobbering low level measurements by "noise".
Back in my day we wouldn't put a switcher in same chassis with sensitive analog stuff.

First thing i'd try is see whether you can replace the offending power supplies with "Linear" type.

I'm recording voltage at 12000hz
Is that the frequency of interest, or your sample rate?

Also back in my day we sometimes used "Rockland" filters to clean up interference
http://www.amazon.com/Wavetek-Rockla.../dp/B0064SGFCY
they were just an excellent adjustable filter with sharp cutoff.

Shielding - the basic premise is to build a Faraday cage around the signal source, masuring system and the cables in between.

i'm an old hardware guy. There's plenty of DSP filtering techniques too, and i'm not qualified in that field.
DanatAMFL
#5
Dec20-12, 10:12 AM
P: 7
The power supplies that I was referring to are very large high power (on the order of a MegaWatt) power supplies that power the facility I am testing in. They can't be avoided. The lines from the sensor to the data acquisition system are relatively long and run past them. The facility is full of other powered equipment and instrumentation, so the noise I'm afraid may be unavoidable. I may look into sheilded wires, however they are expensive and I would need about 16 - 17 of them (I'm actually aquiring multiple signals from multiple sensors). I'd like to find a technique for evaluating the data before I spend the money on trying to physically reduce the noise. I can't show actual data from the facility, but I made a small plot to show you what I'm trying to accomplish.
DanatAMFL
#6
Dec20-12, 10:13 AM
P: 7
12000 hz is my sample rate.
AlephZero
#7
Dec20-12, 02:09 PM
Engineering
Sci Advisor
HW Helper
Thanks
P: 6,948
You could try putting the D/A converter as close as possible to the sensors. Digital data transmission over a "long" distance should be more immune to noise than analog.

If that isn't an optipon, RF shielded cable for signals of a few kHz shouldn't be expensive - try an audio microphone cable before you spend money on something exotic.
yungman
#8
Dec20-12, 02:38 PM
P: 3,883
Fixing the noise is not necessary expensive. What you are dealing with is very common and there are a different kind of techniques to cut the noise down. You don't need to change power supply. I deal with micro volt level signal and I specifically use switching power supply because it's much smaller. I have no issue with noise when I do it right. Why don't you show the schematics, take a few pictures before you decide that digital filtering is the way to go.

I think you are making things difficult by making the wrong assumption. With noise 1/3 the amplitude of the signal, good luck in filtering it out reliably with only digital.

ON TOP, if your have a bad front end design, noise level and characteristics change with slight movement. Even you manage to fix it digitally, tomorrow, the problem might appear again. You have to fix it at the source. You move the wires, noise change and you're screwed again. Fix the problem at the source.
jim hardy
#9
Dec20-12, 05:23 PM
Sci Advisor
PF Gold
jim hardy's Avatar
P: 3,515
That's the trouble with rate detection(differentiation) it's a great noise detector.

By "squared" do you mean:
that you have arithmetically squared its value, as in RMS conversion,
or
that you have simply triggered a bistable to make a pulse with a square shape?

Looking at that graph you made
it looks to me like the question is not
"do i have slope> x"
but
"has my slope greater than x persisted for >y microseconds"

old jim
Mike_In_Plano
#10
Dec29-12, 06:35 PM
P: 560
If the only option is to accept the corrupted signal, you'll always have some measure of difficulty when processing.

For short spikes, I find the following algorithm often helps:
1. Accumulate N samples, x[n], where N is an odd value (i.e. N=15)
2. Sort the values in either ascending or descending order
x[n] -> y[n], where y[n] < y[n+1] for n over 1 to N-1
3. Average the middle values of y[n] and that is your result.
(i.e. for N=15, result = (y[6]+y[7]+y[8]+y[9]+y[10]) / 5)

This works by clipping the highest and lowest values (from noise peaks) off.

If your signal is a tach pulse, and your not needing a rapid response an fft or phase locked loop can pull the signal free from the noise.
NascentOxygen
#11
Jan1-13, 10:03 PM
HW Helper
Thanks
P: 5,145
Quote Quote by DanatAMFL View Post
The lines from the sensor to the data acquisition system are relatively long and run past them. The facility is full of other powered equipment and instrumentation, so the noise I'm afraid may be unavoidable. I may look into sheilded wires, however they are expensive and I would need about 16 - 17 of them (I'm actually aquiring multiple signals from multiple sensors).
Perhaps you could run one additional line, but which carries no data. It would provide you with the noise signal, and you could subtract that from the signal on each data line. Should be easy enough to test to see whether it will prove successful.
Enthalpy
#12
Jan3-13, 01:04 PM
P: 661
Quote Quote by NascentOxygen View Post
Perhaps you could run one additional line, but which carries no data. It would provide you with the noise signal, and you could subtract that from the signal on each data line. Should be easy enough to test to see whether it will prove successful.
Called a differential line. Transmission does not need a symmetrical signal, though it's often done so.

Cables exist with a differential pair plus shielding around both. Useful if strong currents flow in the shield.

Anyway, 0.2V stray if too much. Either this cabling is bad and should be improved, or the power supply is wrongly designed, but this is harder to improve.


Register to reply

Related Discussions
Trying to use LMS adaptive filtering to remove noise with a reference signal Electrical Engineering 0
Data smoothing/filtering General Math 6
Filtering a signal? General Engineering 2
Signal filtering in digital transmission Calculus & Beyond Homework 0
Filtering railway signal Electrical Engineering 0