Horizontal Noise Stripes in an Image

[ecastro]

What are the possible causes of the horizontal noise stripes in this image:

http://dsp.stackexchange.com/questi...odic-noise-from-image-using-fourier-transform

I'm currently studying on these kinds of noise, and how to eliminate them. Can these be caused by diffraction?

Thank you in advance.

 

[tech99]

What are the possible causes of the horizontal noise stripes in this image:

http://dsp.stackexchange.com/questi...odic-noise-from-image-using-fourier-transform

I'm currently studying on these kinds of noise, and how to eliminate them. Can these be caused by diffraction?

Thank you in advance.

The effect is like turning the exposure up and down at 60Hz. Maybe you can identify the frequency and then apply fluctuating gain to the image. In other words, multiply the samples by a 60Hz correction signal. This is different to just subtracting the ripple as an interferer, which will create artifacts. The fluorescent light ripple actually modulates the image .

 

[ecastro]

The fluorescent light ripple actually modulates the image .

How does this modulates the image?

And is it only explicit on the exposure, and is unrelated to diffraction patterns?

 

[tech99]

How does this modulates the image?

And is it only explicit on the exposure, and is unrelated to diffraction patterns?

During the exposure of the frame, the lighting of the scene is going up and down at mains frequency due to the lighting (actually at twice the mains frequency). So to correct this, the gain has to be turned up and down in sympathy.
I don't think you are seeing a diffraction pattern. However, I am not sure in what sequence a digital camera samples the pixels.

 

[sophiecentaur]

Those stripes are a bit difficult to understand fully. If you assume a normal raster-type scan I would have thought that a rolling shutter blind would produce sloping and not horizontal bars from 60Hz mains ripple. For that number of stripes (6?), I reckon the frame rate would have to be 20Hz. Is that right?
To eliminate the stripes, if due to mains hum, it would be necessary to subtract not just a 60 Hz sine wave but the particular waveform of the light variation (squarish, possibly). That could be found for uniformly bright parts of the picture, scaled appropriately and then applied everywhere. Actually, the variations of illumination will affect the whole picture proportionally to the reflectance of the objects in the scene so the right correction function would probably be to multiply by a factor which is (1/x) where x the variation at the brightest parts of the image - rather than subtracting a signal. I haven't thought this our completely but a simple subtraction is not what's needed, I'm sure. tech99 is making the same point, I see.

 

[ecastro]

How about if I use Fourier Transform?

 

[sophiecentaur]

How about if I use Fourier Transform?

When there is no 'side information' then FT may be the only way. I guess you do have the magic values of 120, 240 and 360Hz vertical frequencies to help you. But it is always hard to decide what you want and what you don't want from the original signal in situations like this one. Using a time domain treatment can be very successful if you know the actual waveform you are trying to suppress. A narrow vertical grey level card down the side of every scene will tell you the vertical waveform that's giving you problems. A separate feed from a light sensor could also do it but you would have a problem getting that information into your processor, I guess.
Good news is that the effect on the output image has been to suppress those horizontal bars but also the low frequency contrast is being hit (?). Is there a white/black level re-scaling still needed? I don't speak Python and I can't see what the program does. (Pathetic that my programming is largely pre-object based coding!) Are you also making any corrections at harmonic frequencies? The horizontal line on the bottom step seems to have been suppressed a lot.

 

[ecastro]

Well, this isn't exactly the image I will be correcting. The problem I have is that these kinds of noise may appear in an image, but with different frequencies, i.e. they're not equally spaced from each other. What would be a good method for this?

 

[sophiecentaur]

Well, this isn't exactly the image I will be correcting. The problem I have is that these kinds of noise may appear in an image, but with different frequencies, i.e. they're not equally spaced from each other. What would be a good method for this?

If you can identify this interference as being due to the lighting then you have, potentially, the possibility of eliminating most of it - as long as it doesn't vary much over each line, if you can get a reference signal from somewhere - as I suggested, that could be from a separate sensor or from a reference grey strip in the scene.
Your brief seems to have expanded a bit ("noise may appear in an image, but with different frequencies,"). If it is due to strange lighting supplies then you have the above possible solution.
Basically, if you know the details of the interfering signal OR the wanted signal, then you have a chance of separating them. If not, you have a problem that can only be solved 'to an extent'. Is this interference due to lighting or not?
Do these stripes move up or down the screen with time and does the image that you want have a lot of movement in it? If the interference is 'running through' a stationary image then you could identify the unwanted moving parts.
This is really the Philosopher's Stone of Image enhancement that you are after. Haha - not much to ask.

As your system is digital, the only possible source of your interference must either be from the lighting or the power supply / reference signal to the ADC in the camera - and that's unlikely, I think. Can you change the bars by altering the light source (going outside and using sunlight, for instance - or using DC lighting). Best sort out a few fundamentals of what you are seeing before you waste time on correcting for the wrong causes.

 

[Delta Kilo]

It does look very much like a flicker from fluorescent lighting caused by rolling shutter. This is a well known issue with CMOS cameras. With rolling shutter there is a wave going down through the sensor which removes reset from the pixels, enabling them to accumulate charges, followed by another wave during which the charges are sampled. Distance between waves varies with exposure but the speed is determined by data transfer rate and is usually constant.

Many sensors (like those in mobile phones) have built-in flicker detection and cancellation option in the op-chip processor. Some have explicit 50/60Hz flicker frequency setting, although it may be difficult to get to through the UI of whatever device you are using.

You can try fiddling with exposure, frame rate etc if you can, you might hit a combination where flicker is less pronounced, although image quality might not be ideal.
If you are dealing with continuous video feed and the background is mostly static, you might be able to extract flicker waveform from a series of consecutive frames but it is not going to be easy.

 

[sophiecentaur]

It does look very much like a flicker from fluorescent lighting caused by rolling shutter. This is a well known issue with CMOS cameras. With rolling shutter there is a wave going down through the sensor which removes reset from the pixels, enabling them to accumulate charges, followed by another wave during which the charges are sampled. Distance between waves varies with exposure but the speed is determined by data transfer rate and is usually constant.

Many sensors (like those in mobile phones) have built-in flicker detection and cancellation option in the op-chip processor. Some have explicit 50/60Hz flicker frequency setting, although it may be difficult to get to through the UI of whatever device you are using.

You can try fiddling with exposure, frame rate etc if you can, you might hit a combination where flicker is less pronounced, although image quality might not be ideal.
If you are dealing with continuous video feed and the background is mostly static, you might be able to extract flicker waveform from a series of consecutive frames but it is not going to be easy.

That's an interesting suggestion and involves technology that is not so familiar to your jobbing EE.
But the impairment is sooooo bad. Is it likely that a sensor with that sort of performance would be actually on sale? It is really dire!

 

[Delta Kilo]

In Scotty's voice: "Capt'n, ye cannae change the laws of physics!" If the illumination really does go up and down there is only so much sensor designers can do.
Common solution involves setting the exposure in either 1/100 or 1/120 increments (depending on which country you are in), this way each pixel is exposed for integer number of half-periods. As I said, some cameras do that automatically, some have manual setting but it may not be easy to get to.

But flicker is not the worst of it If there is any motion in the scene (either the object or the camera), you get corresponding geometric distortions, which can be quite severe for fast moving objects. And pretty much nothing can be done about it.

 

[sophiecentaur]

In Scotty's voice: "Capt'n, ye cannae change the laws of physics!" If the illumination really does go up and down there is only so much sensor designers can do.
Common solution involves setting the exposure in either 1/100 or 1/120 increments (depending on which country you are in), this way each pixel is exposed for integer number of half-periods. As I said, some cameras do that automatically, some have manual setting but it may not be easy to get to.

But flicker is not the worst of it If there is any motion in the scene (either the object or the camera), you get corresponding geometric distortions, which can be quite severe for fast moving objects. And pretty much nothing can be done about it.

The artefacts of scanning (bars, movement distortion, moire) have been with us ever since Logie Baird and modern lighting doesn't help but we don't often see the extremes of the OP image in broadcast (non-studio sources). I guess the camera that we're seeing through is not the best quality (?)
It can't be that much of a problem with broadcast equipment or a different form of non-sequential scanning would have been adopted to break up those bars. This stuff has been addressed many times and in many different contexts over the years. Compensation for lighting flicker need not be too much of problem if it's addressed in the system design stages.
It would be interesting to hear from anyone who knows about the practicalities of dealing with difficult light sources.
At least we can probably look forward to further improvements in the quality of LED lighting. They seem to be getting the spectrum a lot better (continuous, at least) and it may only be a matter of time before the flicker can be sorted out. The strobing effect from current LEDs, with moving machinery and ripples on liquids can be 'novel' if not actually annoying.

 

[ecastro]

I'm a bit lost... I'm not really an expert on image processing.

So, what I understand is that the noise is most likely from the lightning of the scene, not the camera? And what if I have a dark edge in the image with similar noise behaviour, and I know that this dark edge should be completely black, how can I use this to correct the image?

 

[sophiecentaur]

I'm a bit lost... I'm not really an expert on image processing.

So, what I understand is that the noise is most likely from the lightning of the scene, not the camera? And what if I have a dark edge in the image with similar noise behaviour, and I know that this dark edge should be completely black, how can I use this to correct the image?

It would have some effect by 'sitting' the black level of each line at the same value. But that would also 'sit' the white level down or up by the same constant amount. What's needed is to alter the gain of the signal along each line. This will produce the right luminance value of each pixel to agree with the pixels immediately above and below. Required pixel value should = reflectance of object X illumination . If the illumination is changing with flicker, you need to compensate 'proportionally, rather than just adding or subtracting some value from each pixel value.
Digital TV is better behaved than analogue because, once the signal is digitised, you don't have the problem of gain changes and added hum etc. that were so common in the old days.
I still don't see how sensors can be so bad as the OP is showing. The scene is pretty non-critical (?) and, if it were a common problem, we would see the effect on our TV screens every day. Are we dealing with 'bottom of the range' equipment here? (Or just an old type of digital camera).

 

[Delta Kilo]

It is a combination of both. The light flickers and the camera exposes different scanlines at different times.
http://www.diyphotography.net/everything-you-wanted-to-know-about-rolling-shutter/

[Sorry just noticed your post where you said your noise is not equally spaced. ]

Well, this makes it more complicated. Still, horizontal bands would indicate noise signal in low audio frequencies range. It may be either optical (flicker, flash photography, lightings) or electrical but the latter is pretty much limited to badly designed analog power supply in the camera which is unlikely. There are also some pure digital effects that can produce banding (frame tearing due to badly done frame rate conversion and aliasing due to badly done resampling) but they are quite distinct in their appearance. Anyway, hard to tell without looking at the image.

A good method to remove the noise uses as much information about the noise and the image content as you can get. There is no universal method that works for every case. Can't get any more specific without seen the image. But just to give an example: In your case the bands are horizontal. You can project the image onto Y axis (average your image in X direction) then subtract the projection from your image. You will get an image which is going to look flat, but will have all the fine details preserved (actually this is exactly what the guy on stackexchange did by mistake, you will notice all scanlines in the "output image" have the same average value)\. But what is important - all the noise is now concentrated in Y projection and none of it in the rest of the image. But of course there is also useful information in it which you'd want to preserve. Still, it might be easier to identify and remove the noise from Y projection and then re-apply it to the image. For example, in stackexchange image, a plot of Y projection will show a clear sine wave superimposed on it. If you remove it and add the modified profile back, you should get a reasonably good result. Incidentally this is exactly equivalent to doing an FFT, setting small areas around 2 specific points in the spectrum to 0 and converting it back, as was suggested by someone on stackexchange.

And yes, you can use known reference features. The easiest way to do it is if you can get a series of images from the same viewpoint, then you can average them to create a noise-free reference background image. You can then subtract it from your new image and the difference will contain just the noise plus whatever action is happening in the foreground. Then you can extract the noise pattern from the known background portions and somehow expand it it to the rest of the image.
BTW intensity values of "completely black" features are not very reliable. Uniformly colored flat bright features are much better for these kind of things.