What are some techniques for filtering and amplifying pulse signals?

[NascentOxygen]

What is the fastest repetition rate of the pulses that will be encountered? You are using 50kHz here, is that representative? Is the baseline drift that you are trying to remove just mains hum, i.e., 60Hz?

You could build a monostable that triggers on the rising edge, this will indicate the +ve going pulses. Invert your analog signal and parallel feed it to a second identical monostable, the output of which will then indicate the presence of -ve going pulses.

 

[jim hardy]

What's the trick to find similar opamp from different brands? Just read a bunch of datasheets until you find something that looks similar?

I find manufacturer's selection guide and look for similar supply voltage, input offset & bias, and unity gain BW.

Op-amps are getting so good it's hard to find ones as bad as I grew up with. LM324 was improvement over 741 which was improvement over 709 , and it is looking anemic now.
You young guys are amazing to me - you're not overwhelmed by so much progress and the myriad gizmos available. I guess that's why we are allotted but threescore and ten - world changes too much for us.

...but how should the direction of the pulse be identified?

Thought experiment time

Let the tail wag the dog.

Once you have them centered about zero,

IF you trigger on only one transition... and examine level shortly after that trigger..

A positive pulse has, shortly after its positive transition, a positive value
A negative pulse has, shortly after its positive transition, a zero value

Sounds like a setup for an edge triggered monostable delay (74123 or 555) and D flip-flop(7474?).

It's late - sanity check me please..

 

[petterg]

What is the fastest repetition rate of the pulses that will be encountered? You are using 50kHz here, is that representative? Is the baseline drift that you are trying to remove just mains hum, i.e., 60Hz?

You could build a monostable that triggers on the rising edge, this will indicate the +ve going pulses. Invert your analog signal and parallel feed it to a second identical monostable, the output of which will then indicate the presence of -ve going pulses.

It's about .2ms between pulses. That makes 5kHz.
I'm want the pulses to start from 0. That is, remove the slowly changing offset. It's slower than 1Hz.

The challenge is that both the offset and the pulses changes amplitude with a factor of 500. That is, at some point the offset may be -0.3mV and pulse peak may be +1mV. At some other point the offset may be +150mV and puls peak may be +650mV. (That was for positive pulses. For negative pulses all numbers will change sign.) So the challenge with monostables is to trigger on the pulse, not the changing offset. The differentiator finds edges nicely, but it finds two edges for each pulse. If there is a way to remove / ignore the second edge, it would be perfect for triggering a couple of monostables.

 

[petterg]

Once you have them centered about zero,

IF you trigger on only one transition... and examine level shortly after that trigger..

A positive pulse has, shortly after its positive transition, a positive value
A negative pulse has, shortly after its positive transition, a zero value

Sounds like a setup for an edge triggered monostable delay (74123 or 555) and D flip-flop(7474?).

It's late - sanity check me please..

I see half of your idea. How would you avoid that the falling edge of a positive pulse trigger the monostable as a negative pulse?
Two monostables, one trigger on rising edge, one on falling. They both are hooked up so that they don't respond to anything if any of them are in unstable state. If the monostables reminds in unstable state for 50us, then they would ignore the edge following close after the last edge. Hence ignore 2nd edge of pulse.
This sounds like something that could work?

 

[jim hardy]

Edge triggered in my day meant it triggers on either a rising or a falling edge but not both.

Consider a negative going pulse.
Signal starts at zero, transitions negative to a non-zero value and holds a while, then transitions positive back to zero.
Just after its positive transition it has value zero. So if we look at it then and find zero, we know it was a negative pulse. We can ignore its negative transition

Positive pulse :
Signal starts at zero, transitions positive and holds a while then transitions negative back to zero.
Just after its positive transition it has value not zero. So if we look at it then and find nonzero we know it is a positive pulse.

That's the simplest, to look at transitions one direction only, but with your varying levels might be error prone.

Your dual approach is probably better -
with two data bits you can establish four states:
positive pulse,
negative pulse,
no pulse at all,
and 'error' or indeterminate.

Here's the sort of building block gizmos we had in my day:

http://www.ti.com/lit/ds/symlink/sn74123.pdf

http://www.electronics-tutorials.ws/waveforms/monostable.html

http://www.learnabout-electronics.org/Oscillators/osc45.php

old jim

 

[petterg]

Edge triggered in my day meant it triggers on either a rising or a falling edge but not both.

Consider a negative going pulse.
Signal starts at zero, transitions negative to a non-zero value and holds a while, then transitions positive back to zero.
Just after its positive transition it has value zero. So if we look at it then and find zero, we know it was a negative pulse. We can ignore its negative transition

How do you make it ignore?

The differentiator show the rising side of the (positive) pulse as a negative value, the following falling side as a negative value. Hence the result has a falling edge, a rising edge, and another falling edge for each pulse. An edge triggered device after the differentiator should only trigger at one of those edges (doesn't matter which one). How can it be configured to only trigger on one edge?
I was thinking of two 555-based monostables, holding unstable state for 50us, one having inverted input. And connect output from each to the reset on the other. First edge would then trigger one of the 555's, which pulls the reset on the other 555. The second edge would trigger the second 555, but it will be ignored because the reset is held. Third edge would then bring the trigger signal back to 0 and not make any change as the first 555 had not timed out yet. Then the first 555 would time out and it's ready for next pulse in any direction... it all sounded so well, so I tried with the attached schematic. Output from U4 and U5 would trigger each of the 555's.

To make it work with small signals I had to include U3 to amplify the input.
So the circuit is:
U1: input high impedance buffer
U3: input amplifier
U2: differentiator
U4/U5: (inverting) amplifier (555 input will require at least 800mV from vGND to trigger.)

This seemed to give a reasonable signal to the 555's, when input signal was weak (the lower graphic). However, when the input was stronger, clipping occurred. The clipping itself doesn't matter, but the 3rd edge is so late that the reset is released before the signal gets below trigger level of the 555's.
If I increase the 555 timeout to more than 50% of the cycle, I see a risk that it will trigger on 2nd edge, and ignore 1st edge on the next cycle.

 

[meBigGuy]

I haven't kept up completely, So I am not sure what the blue and green "smooth" signal are and where they came from in the circuit. Are blue and green the two possible inputs, and the smooth signal is the filtered possible outputs? I'm also not sure to what extent these two signals represent ideal or worst case levels.

Can you summarize it for me, and maybe I can think of a cool way to detect them.

 

[petterg]

Blue / green is the output from U4 / U5.
The upper graphic is when input has the highest expected amplitude (both pulse and slow sine)
The lower graphic is when input has the lowest expected amplitude (both pulse and slow sine)

Outside the picture, on the left is a function generator and a transformer. From input signal passes through a high impedance buffer (U1), is amplified (U3), passes through the differentiator (U2) and is amplified again so that even the weakest expected signal will be strong enough to trigger a monostable based on a 555.

I'm currently reading up on automatic gain control with the intention to reduce gain in one or both amplifier stages. So far I'm leaning more towards just amplify the input signal, feed it to mpu and do the rest digitally. (= I'm about to give up analog processing.)

 

[meBigGuy]

One more question. Relative to the Blue signal, what it is you want to detect (I sort of lost track of the original + and - pulses)?

 

[NascentOxygen]

It's about .2ms between pulses. That makes 5kHz.

Is that a dependable fixed unvarying time period?

 

[petterg]

One more question. Relative to the Blue signal, what it is you want to detect (I sort of lost track of the original + and - pulses)?

MPU need to work with the following 3 states:
Pulses is positive
Pulses is negative
No pulses (no signal or signal to weak to be detected)


Edit:
Relative to the graphics, a pulse starting on T=0. I want to detect a pulse and it's direction. Delay doesn't matter. So I want to detect the cycles of blue or green, and I need to know which of them to pay attention to. As I pictured above, using the blue and green to trigger one 555 each. And say the 555 has a hold time of 50% of a cycle. The 555's will trigger when input is below Vcc/3. That is 5V/3 = 1.667V. Vgnd = 2.5V. Hence the 555 will trigger when signal is more than 1.667V-2.5V=-0.833V.

So, considering the weak signal graphed above, the timeline looks like this:
T=0, no triggers. Both 555's are recently reset.
T=20: blue will trigger 555#1.
T=20+[delay]: 555#1 will trigger and hold reset on 555#2
T=40: blue will release trigger on 555#1
T=80: green is close to trigger 555#2. A little bit stronger signal, or some noise could be enough to trigger. But 555#1 is still holding the reset of 555#2, so it doesn't matter if the trigger is pulled or not.
T=120: 555#1 times out and returns to stable state. Reset on 555#2 is released.
T=200: new cycle starts
T=220: blue will trigger 555#1.
...

This was the positive pulse. With negative pulse the blue and green would swap, hence 555#2 would be triggered and 555#1 would be held in reset.

But there's a risk of this system will be out of sync when powered on. Say it's powered on at T=60, and assume reset of both 555's is part of the power on procedure. Then timeline will look like this:
T=80: green is close to trigger 555#2. A little bit stronger signal, or some noise could be enough to trigger. Assume it's triggered. (No trigger will just let the circuit wait for start of next cycle.)
T=80+[delay]: 555#2 will trigger and hold reset on 555#1
T=85: green will release trigger on 555#2
T=180: 555#2 times out and returns to stable state. Reset on 555#1 is released
T=200: new cycle starts. Both 555's are ready
T=220: blue will trigger 555#1.
T=220+[delay]: 555#1 will trigger and hold reset on 555#2
T=240: blue will release trigger on 555#1
T=280: green is close to trigger 555#2. A little bit stronger signal, or some noise could be enough to trigger. But 555#1 is still holding the reset of 555#2, so it doesn't matter if the trigger is pulled or not.
T=320: 555#1 times out and returns to stable state. Reset on 555#2 is released.
T=400: new cycle starts
T=420: blue will trigger 555#1.
...

So the out of sync-issue will make one error before it recovers to the correct sync.

For the strong signal, the timeline looks like this:
T=0, no triggers. Both 555's are recently reset.
T=2: blue will trigger 555#1.
T=2+[delay]: 555#1 will trigger and hold reset on 555#2
T=50: blue will release trigger on 555#1
T=52: green will trigger 555#2, but 555#1 is still holding the reset of 555#2, so it doesn't matter that the trigger is pulled.
T=102: 555#1 times out and returns to stable state. Reset on 555#2 is released.
T=102+[delay]: green is still holding the trigger of 555#2. Hence 555#2 is triggered.
T=102+2x[delay]: 555#2 pull the reset on 555#1
T=170: green will release trigger on 555#2
T=200: new cycle starts
T=202: blue will trigger 555#1, but it won't react as 555#2 is holding it's reset
T=203: 555#2 times out
T=203+[delay]: 555#2 releases reset on 555#1
T=203+2x[delay]: 555#1 is triggered
...

So, a strong signal makes the 555's fail totally.

Increasing the timeout for the 555's may help some, but it will lose the functionality to recover when out of sync.

 

[petterg]

Is that a dependable fixed unvarying time period?

I can change the time period if it makes things easier. It will be fixed on what ever value I decide it to be. Pulse width will be between 1/3 and 1/5 of the period.

 

[NascentOxygen]

I can change the time period if it makes things easier. It will be fixed on what ever value I decide it to be. Pulse width will be between 1/3 and 1/5 of the period.

I don't suppose that that underlying clocking signal that's part of the transmitter can be made available at the receiving (demodulator) end?

 

[petterg]

I don't suppose that that underlying clocking signal that's part of the transmitter can be made available at the receiving (demodulator) end?

By (a second) radio it could be possible. Then it would need to account for radio wave reflections. I think it will be to complicated.

(Note I did a large edit 3 posts up)

 

[NascentOxygen]

By (a second) radio it could be possible. Then it would need to account for radio wave reflections. I think it will be to complicated.

Is there a pulse every clock cycle, or are there some cycles when a pulse is not generated?

 

[petterg]

Is there a pulse every clock cycle, or are there some cycles when a pulse is not generated?

Pulse will always be generated (unless power is lost at sender). But there will be situations where no pulse is received (signal too weak). However, if power is lost at sender, receiver will need to be able to resume operation when power to sender returns.

 

[petterg]

Actually, if the monostable (555 or other) timer starts when trigger is released, I think the problem is solved.

The monostable needs the following behavior:
- When reset is triggered: goto stable state no matter input signal
- When reset is held: stay in stable state no matter input signal
- When input is triggered: goto unstable state (unless reset tells otherwise) (555 starts timer at this point)
- When input is held: remain in unstable state (unless reset tells otherwise)
- When input is released: start timer
- When timeout: return to stable state
- When input is (re)triggered while in unstable state: reset timer (555 will not reset timer when retriggered)

Are there any know monostable out there with this behavior?

 

[NascentOxygen]

Pulse will always be generated (unless power is lost at sender). But there will be situations where no pulse is received (signal too weak). However, if power is lost at sender, receiver will need to be able to resume operation when power to sender returns.

The only confounding noise you have mentioned is a low-frequency baseline drift. Is it safe to assume that there is no other interference of significance? The problem with a differentiator is that it exacerbates pulse noise. Can we assume this is not likely to be a problem here?

 

[petterg]

I'm sure there will be noise. How much, and what to do with it is something I'll have to figure out if it becomes an issue. The noise will for sure have a higher frequency than the drifting offset. The offset is as close to a DC a signal can ever become without being a DC. It should the the easiest part to filter out, still it seems to be impossible to filter. Hence any noise will be worse than impossible to filter. I'm looking forward to that. First step should be to have a circuit working without noise.

 

[NascentOxygen]

An outline of the way I'd approach this. I'd look at using a PLL chip to regenerate that 5kHz clocking signal (or at least a squarewave synchronous with it). Because each transition of that timing squarewave co-incides with the pulse position, sample your analog signal at that time using a sample-and-hold. Compare the amplitude of that sample with the analog signal a moment later, just beyond the pulse position. From their relative amplitudes deduce the polarity of that pulse.

EDITED

 

[meBigGuy]

Can you post a picture of positive pulses followed by negative pulses at the min and max levels.

Unlikely possibility 1:
It seems like you can trigger on the rising edge of both, which effectively becomes the leading edge of the positive pulse and the training edge of the negative pulse. Like putting a comparator at 0.4V and using it to fire the 1 shot. I expect when positive follows negative and vice versa it won't be so clean though.

Possibility 2:
At open loop gain op-amps make good (although slow) comparators. Use +- 0.8 volts as the references and deal with the "digital" output of the final two opamps.

Make a "short pulse" detector. If a positive pulse fires timer1, it enables a gate to fire timer2 if the pulse drops before timer1 times out. Do the same thing with the negative side.

Did that make sense?

BTW, using an ADC and figuring it out in the MPU is a good idea if you have the cycles.

 

[petterg]

An outline of the way I'd approach this. I'd look at using a PLL chip to regenerate that 5kHz clocking signal (or at least a squarewave synchronous with it). Because each transition of that timing squarewave co-incides with the pulse position, sample your analog signal at that time using a sample-and-hold. Compare the amplitude of that sample with the analog signal a moment later, just beyond the pulse position. From their relative amplitudes deduce the polarity of that pulse.

EDITED

I don't think I fully understood your idea. Regenerating signal will be easy. In order to have any further use of it it has to be in sync. Making sync will require signal amplitude within a certain range from the original signal. Wouldn't that just be a new angle to run into the same issue?

 

[petterg]

Can you post a picture of positive pulses followed by negative pulses at the min and max levels.

The swap from positive to negative (and back) happens by amplitude going gradually down to 0, then increase gradually. There will be at least 1000 pulses from max positive to max negative.

Unlikely possibility 1:
It seems like you can trigger on the rising edge of both, which effectively becomes the leading edge of the positive pulse and the training edge of the negative pulse. Like putting a comparator at 0.4V and using it to fire the 1 shot. I expect when positive follows negative and vice versa it won't be so clean though.

For the incoming signal I can trigger at any edge. After the differentiator there may be too many edges. Trigger has to be on leading edge. If you're thinking of a one shot after the differentiator, how would it know which edge to trigger?
If you're thinking of a one shot on the original (amplified) pulse, there is the issues of amplitude range from weaker than trigger level to stronger than clipping. Clipping causes multiple trigger edges. How would the one shot go clear of that?

Possibility 2:
At open loop gain op-amps make good (although slow) comparators. Use +- 0.8 volts as the references and deal with the "digital" output of the final two opamps.

Make a "short pulse" detector. If a positive pulse fires timer1, it enables a gate to fire timer2 if the pulse drops before timer1 times out. Do the same thing with the negative side.

Did that make sense?

I though so, until I tried to sketch some blocks and realized I was lost.

BTW, using an ADC and figuring it out in the MPU is a good idea if you have the cycles.

Looks like it. I still can't believe suppressing a slow sine should be this complicated thou.

 

[meBigGuy]

I am assuming that "differentiator" means high pass filter. I am looking at the output (blue and green) waveforms and assuming that if I looked at 1 output that is what I would see for positive and negative. Maybe I need to see the input to the two final amps.

Background:
Do you understand how you can use an opamp as a comparator. Put 0.4V DC reference on the minus input (with no feedback resistor). When the plus input goes above 0.4 the output slams to the positive rail, and vice versa. You can do tricks like make the threshold a fraction of the peak voltage (with a time constant for attack and decay). If you use an actual comparator chip the output goes from 0 to +V (sometimes determined by an output pullup resistor) which is easier to work with than -V to +V. A comparator converts analog domain to digital domain.

Regarding Method 1
Observe all the blue and green positive crossings at +0.4V. There are positive crossings from the leading edge of positive pulses, and positive crossings from the trailing edges of negative pulses. If you triggered on those (after a comparator), you have total pulses, but still need a way to determine whether it was a positive or negative input pulse. Not sure of the best way to determine that.

Regarding Method 2: Just think about positive crossings at +0.8V for now, then you can apply the same logic for negative crossings at -0.8V.

The goal is to trigger an output pulse when there are short pulses. Basically fire a timer on the positive edge, and then fire another timer on the negative edge if the first timer is still true.

Fire timer 1 which is X ms (always longer than the pulse). Logically AND the inverse of the input pulse with the output of the timer and use that to fire timer 2. When the input pulse goes low it fires timer 2.

Regarding AGC:
What is making this hard is there is no gain control. If the gain varied such that you always got the lower picture, this would be easy. With your slow variations, buuilding in AGC would be easy. If you did it digitally, that is essentially what you would do.

 

[petterg]

I am assuming that "differentiator" means high pass filter. I am looking at the output (blue and green) waveforms and assuming that if I looked at 1 output that is what I would see for positive and negative. Maybe I need to see the input to the two final amps.

A differnetiator is similar to a highpass, but different. It does the opposite of an integrator. In post #30 you see the input (green) to the differentiator and its output (blue).
https://www.physicsforums.com/showpost.php?p=4500465&postcount=30

Input to the final two opamps is equal to the output of the differentiator, just amplified more to handle the weak signals (which causes clipping on strong signals).

Background:
Do you understand how you can use an opamp as a comparator. Put 0.4V DC reference on the minus input (with no feedback resistor). When the plus input goes above 0.4 the output slams to the positive rail, and vice versa. You can do tricks like make the threshold a fraction of the peak voltage (with a time constant for attack and decay). If you use an actual comparator chip the output goes from 0 to +V (sometimes determined by an output pullup resistor) which is easier to work with than -V to +V. A comparator converts analog domain to digital domain.

I know the comparator behavior. But I don't see how comparing to a fixed level will work, unless passed through AGC first. The variable offset of the input will cause trouble. That's back to the idea I wrote in the first post, as highpass didn't work, do a low pass, and compare/subtract the lowpass filtered signal to the original. Maybe combining the lowpassed signal with an fixed offset with a sign decided by last pulse direction, and use that combined result to compare to the original would do better. Don't know how to combine them in a way that makes sense thou.

Regarding Method 1
Observe all the blue and green positive crossings at +0.4V. There are positive crossings from the leading edge of positive pulses, and positive crossings from the trailing edges of negative pulses. If you triggered on those (after a comparator), you have total pulses, but still need a way to determine whether it was a positive or negative input pulse. Not sure of the best way to determine that.

The hole goal is to determine if the pulses are positive, negative, or none. The pulses them self doesn't really matter. All I need to know is if there are any pulses, and if so which direction do they have.

Regarding Method 2: Just think about positive crossings at +0.8V for now, then you can apply the same logic for negative crossings at -0.8V.

The goal is to trigger an output pulse when there are short pulses. Basically fire a timer on the positive edge, and then fire another timer on the negative edge if the first timer is still true.

Fire timer 1 which is X ms (always longer than the pulse). Logically AND the inverse of the input pulse with the output of the timer and use that to fire timer 2. When the input pulse goes low it fires timer 2.

That sounds like the 555-hold-reset I'm writing in post #40 and #47. That ran into an issue that the timer need to have the ability to trigger it's output on positive edge, don't start timer until negative edge, and retrigger timer on next positive edge. I have a theory that this should be possible to do with a 555 monostable with addition of a transistor in parallel with the timing capacitor. The transistor should be controlled by the input signal, but I haven't succeeded to make it work. The thing is that the capacitor have one pin to ground and a charge between 0V and 3.3V. The idea is to discharge the capacitor trough the transistor when input signal has a voltage of less than 1.7V (or more than 3.3V if using the inverted input). Using voltage dividers that was possible, but discharge became too slow.

Regarding AGC:
What is making this hard is there is no gain control. If the gain varied such that you always got the lower picture, this would be easy. With your slow variations, buuilding in AGC would be easy. If you did it digitally, that is essentially what you would do.

I've tried to look into that. They seem very complex, and schematics uses symbols I've no clue what is. That scares me off in favor of MPU.

 

[meBigGuy]

That sounds like the 555-hold-reset I'm writing in post #40 and #47. That ran into an issue that the timer need to have the ability to trigger it's output on positive edge, don't start timer until negative edge, and retrigger timer on next positive edge. I have a theory that this should be possible to do with a 555 monostable with addition of a transistor in parallel with the timing capacitor. The transistor should be controlled by the input signal, but I haven't succeeded to make it work. The thing is that the capacitor have one pin to ground and a charge between 0V and 3.3V. The idea is to discharge the capacitor trough the transistor when input signal has a voltage of less than 1.7V (or more than 3.3V if using the inverted input). Using voltage dividers that was possible, but discharge became too slow.

You are making it harder than it is. Use a comparator with 0.8V reference and two timers to detect short pulses as I outlined. Then do the same for a comparator with -0.8V reference.

That is a total of two comparators and 4 positive edge triggered one shots. You get two pulse streams, one for positive pulses and one for negative.

 

[NascentOxygen]

I don't think I fully understood your idea. Regenerating signal will be easy. In order to have any further use of it it has to be in sync. Making sync will require signal amplitude within a certain range from the original signal. Wouldn't that just be a new angle to run into the same issue?

I think it would have the ability to correctly detect pulses buried in noise far better than an arrangement without the stability of a closed loop.

But I do think you should direct further effort into filtering out the predominant low-frequency interference. I don't know what you did wrong with your filter tests, but I can't see how there can be any difficulty removing it while preserving the waveshape you seek. I might take a closer look at the task myself.

 

[meBigGuy]

I don't know what you did wrong with your filter tests, but I can't see how there can be any difficulty removing it while preserving the waveshape you seek.

I tend to agree

 

[petterg]

Nothing would be better than using filter. Everything else has been attempts to find a workaround.
Post #16 has the highpass filter.
https://www.physicsforums.com/showpost.php?p=4496667&postcount=16

Do you see why it's not doing any good?

 

[jim hardy]

Post 16:

------------------------------------------------------------------------------------------------------------

Attached is my highpass approach.
Input is wires coming in from the left.
U1A is virtual ground generator.
U1B is an impedance buffer, having a DC block before it.
U1C and the RC's in front of it is the highpass filter.

Ch.A on the scoop is output from U1B. It's almost flat between the pulses. Voltage in the pulse max is about 4 times the level of voltage between the pulses.
Ch.B is output from U1C. It's starting to look more like sawtooth. Voltage in the pulse max is only about double the level of voltage between the pulses. And the voltage between the pulses are stronger than before the filter. So I've managed to get only the negative effects of the filter, none of the positive ones.

That's why I don't think highpass is the way to go.
Next up is lowpass + comparator.

----------------------------------------------------------------------------------------------------------

Do you see why it's not doing any good?

No.

1. Are we really dealing with one millivolt of signal?
2. What's wrong with ch A ? Looks to me like it makes whatever is the input signal into negative going pulses.