A Detection of hot spots with a thermal (non-radiometric) camera

[pcarvalho]

TL;DR

We need to detect hot spots with a thermal (non-radiometric) camera. The camera retrieves JPEG.
We need to know if it makes sense to do the processing on top of JPEG.

Hello,

We have a thermal camera and its purpose is to detect hot spots at different distances. We made an experiment with a JPEG picture and we noticed the following:

• At the same distance, one object at 600 degrees and an object at 38 degrees (human body) have the same pixel intensity (255 in grayscale).
• The image adjusted when the 600 degrees object exited the scene (parts of the human body and background became brighter).

We will make a detection algorithm and we need to make sure only the 600 degrees object is detected. If we are going to do the algorithm based on the maximum pixel value we belive this will create false positives.
So, I want to understand if it makes sense to use JPEG images to do the processing. If it makes sense, what would be the maximum distance that the algorithm would work fine?
If you have an idea, I would like to ask for references/papers that use jpeg to detect objects that are above a certain threshold in terms of pixel intensity and that make the distinction between high temperature objects and low temperature objects.

Many thanks

 

[sophiecentaur]

TL;DR Summary: We need to detect hot spots with a thermal (non-radiometric) camera. The camera retrieves JPEG.
We need to know if it makes sense to do the processing on top of JPEG.

At the same distance, one object at 600 degrees and an object at 38 degrees (human body) have the same pixel intensity (255 in grayscale).

Hi. Welcome to PF.
A diagram of the system would be useful. Many things spring to mind but, for meaningful results, you need to "control all your variables".

What is the 600c source?
I'm not sure what your experiment is trying to achieve but it seems very likely to me that the images are 'clipping' . Both images have 255. That is surely not a coincidence. If the hot object is causing the measurement of the cool object then, either the gain of the system is being varied (auto exposure) or the 600C object may be larger than its 'obvious' image suggests; is the hot object warming up its surroundings?

You need to calibrate your system. Look at images of the background with and without the hot source in place. Do the pixel values vary? How do the pixel values drop off at the edges of the hot object. Plot a graph using the 'colour dropper' for samples against position.
Also, you should use a mask to isolate the body image.

TL;DR Summary: We need to detect hot spots with a thermal (non-radiometric) camera. The camera retrieves JPEG.
We need to know if it makes sense to do the processing on top of JPEG.

If you have an idea, I would like to ask for references/papers that use jpeg to detect objects that are above a certain threshold in terms of pixel intensity and that make the distinction between high temperature objects and low temperature objects.

I have no idea what specific software is available directly for your purpose but there are many image analysis apps which you could use. Have you done any searching for this? Searching is the way through; it's good for the soul and you can't expect PH to do the legwork for you unless a member happens to have done this sort of thing before! However, without that, you can get the RGB /Y values from the colour dropper and plot your own graph along any line across the screen. Whatever operating system you are using that facility will be available on free photo processing apps. A graph (plot many of them - on paper if necessary) would give you a good idea of what's actually happening.
JPEG is fair enough for that sort of work. The artefacts it generates are not likely to spoil your data.

 

[pcarvalho]

Ok, I'm going to give you more insight. We have an hikvision camera (thermal, not radiometric).
The idea is to spot high temperatures at different distances (10+ kilometers). We know that radiometric cameras can measure temperatures, but these temperatureas are apparent. With a formula we could know more or less the temperature based on the distance. We know that these cameras give raw data. Raw data is data from the ADC of the camera, which means that if the camera is 14 bit we will have a range of 0-16384 values.
But our camera is not radiometric, it is only thermal, which means only the jpeg can be extracted. We want to make our own thermal analysis app. Basically we capture a frame from the camera, do the processing (find hot spots at different distances) for each pixel, and show where it is in the image.
We will have an edge computer that captures the frame from the camera, process it and send to a server. This will be done in a 5-10 seconds time frame.
Besides, we can't find anywhere algorithms that detect high temperatures in jpeg frames. However ChatGPT, says it is possible. I tried one of the algorithms of it and the results are bad (several places of the scene are misindentified as hot source). The hot source we used is a kind of dryer for heat-shrink sleeves. It says it can go to 600º.

Thanks

 

[sophiecentaur]

But our camera is not radiometric, it is only thermal

The only difference (as far as I can tell) is accuracy of the measured temperature. My problem is not knowing how temperature can be measured without at least two filters and two sets of sensors in the array. The ratio of two spectral readings at different wavelengths will tell you where you are on the black body radiation curve and, hence the temperature.

Besides, we can't find anywhere algorithms that detect high temperatures in jpeg frames.

I don't understand that statement. JPEG is only a method of converting a source image to a bit reduced version. Are your images Coloured? (Just to be sure that the output is not monochrome). Plenty of software can change back from JPEG to RGB and to any other coding .

The images from even cheap thermo cameras are all coloured. So (I conclude) White corresponds to high temperatures and red is lower temperatures (or what ever false colours they choose)

I just hope that someone in PF can HELP with how the images give temperatures because I can find nothing that's equivalent to the three colour analysis . It's all 'magic' according to the adverts.

 

[sophiecentaur]

for visible imaging.

 

[pcarvalho]

The only difference (as far as I can tell) is accuracy of the measured temperature. My problem is not knowing how temperature can be measured without at least two filters and two sets of sensors in the array. The ratio of two spectral readings at different wavelengths will tell you where you are on the black body radiation curve and, hence the temperature.

I don't understand that statement. JPEG is only a method of converting a source image to a bit reduced version. Are your images Coloured? (Just to be sure that the output is not monochrome). Plenty of software can change back from JPEG to RGB and to any other coding .

The images from even cheap thermo cameras are all coloured. So (I conclude) White corresponds to high temperatures and red is lower temperatures (or what ever false colours they choose)

I just hope that someone in PF can HELP with how the images give temperatures because I can find nothing that's equivalent to the three colour analysis . It's all 'magic' according to the adverts.

The jpeg image is grayscale. We use the "White hot" pallette.
The ideal solution was to have raw data, but this is not possible with the camera we have. We know that jpeg is a compression algorithm (we are losing information).
Also, we are in this trouble because we need to sell a whole system that includes that camera, but this camera was sent to us wrongly by the seller. Now we don't have time to send it forth and back.
Anyway, I'm more leaned that we can't detect well hot spots with this camera, but I just need someone that confirms that. I was hoping someone from universities that use this kind of equipments could give me a confirmation. Maybe this should be asked directly to a professor?

 

[sophiecentaur]

We use the "White hot" pallette.

How does your system distinguish between radiated flux and temperature? You seem to suggest it doesn't. I don;' think there can be any way to change that.

As far as your perceived JPEG problem goes, what sort of resolution are you after? JPEG divides the image into 8X8 pixel blocks and then codes the original variation within the blocks to a bit reduced version which, for most pictures, is indistinguishable from the original. As your images are monochrome the colour limitations of JPEG won't count. So why would you be bothered? How high res are your images? How would raw data help you?

 

[sophiecentaur]

Besides, we can't find anywhere algorithms that detect high temperatures in jpeg frames.

I don't see why you would expect to find references if, as I say, it's not relevant. One pixel in a RAW image will be very close to the value of the same pixel in a JPEG image, particularly in a monochrome image. Don't worry; the fact that it's not reported as an issue implies that is not an issue.

The fact that the cooler and hotter images give the same value in your camera implies that the system gain is too high. If you have a neutral density filter and if it doesn't affect the values of your pixels would mean that you are overloading your system. Try using a smaller aperture until you can see an effect and that will be more like the 'proper' exposure.

 

[pcarvalho]

I don't see why you would expect to find references if, as I say, it's not relevant. One pixel in a RAW image will be very close to the value of the same pixel in a JPEG image, particularly in a monochrome image. Don't worry; the fact that it's not reported as an issue implies that is not an issue.

The fact that the cooler and hotter images give the same value in your camera implies that the system gain is too high. If you have a neutral density filter and if it doesn't affect the values of your pixels would mean that you are overloading your system. Try using a smaller aperture until you can see an effect and that will be more like the 'proper' exposure.

Hi there,
I contacted the supplier of the camera and only today I got an answer. Apparently, it is not possible to switch off the auto gain control.
If I understood correctly about AGC, this feature adapts the intensity of the pixels based on what is in the image. For instance, if we have a human in the scene (38º) and there is no higher temperature object in that same scene, the pixels where the human is will have 255 in intensity. If we put a higher temperature object on the scene, the pixels of the human will have a slightly less intensity, i.e below 255. If we disable the AGC, maybe the human pixels will have lets say 100 in intensity independently if there is a higher temperature object or not on the image.
In sum, if we use the auto gain control feature we will have tons of false alarms which is bad.
But please confirm that what I said about the AGC is correct.

Many thanks!

 

[sophiecentaur]

But please confirm that what I said about the AGC is correct.

I think that's the right idea but surely your AGC should adjust the gain until the hottest object is 255. You report that it doesn't do that.

If the camera adjusts exposure so that the highest pixel is 255 then I could only suggest that you have a reference object in all your pictures and scale the other readings to it. I worry that the pixel reading for both your hot and warm objects are the same. That implies to me that the gain control is not working. What doe the suppliers say about that? Better still, go to the manufacturers and ask the same question; many equipment suppliers don't have a technical clue about what they sell.

Imo you need some practical experience of the camera, with several different images and get a feel for what it is actually doing with its signals. Get really familiar with what it does. Look at the image of an open fridge door, for instance. Does it have a 'dark' image? Look at a building and see what the windows look like. How does the 'dark' level of most of the image change with peak areas? Use a filament lamp with a dimmer or a variable electric hotplate. Please post a couple of pictures; we can then look at some pixel values. That's easy to do as you can probably just drag and drop into your text or do it the long way as described. 'Attach files', if necessary). What is the product name?

I hope the JPEG problem no longer worries you.

 

[sophiecentaur]

@pcarvalho I have been looking at Hikvision cameras on the site and they claim a temperature range uo to 150C. You say your hot object is at 600C (around dull red hot). I'd suggest that the camera is not matched to the task you want it for. Did you already check this before you started?

 

[pcarvalho]

@pcarvalho I have been looking at Hikvision cameras on the site and they claim a temperature range uo to 150C. You say your hot object is at 600C (around dull red hot). I'd suggest that the camera is not matched to the task you want it for. Did you already check this before you started?

@sophiecentaur The system was ordered from a chinese supplier (ptz+cameras) to be sold to our client. The client specified the characteristics of the camera and we asked them (chinese supplier) to grant a system that would work for them. They gave us a PTZ with that camera. Now apparently it doesn't satisfy the requirements and we don't have time to send forth and back the system. Ideally, we would have a thermographic camera and that way it would be way more simplier to integrate it. We asked a camera that we could extract raw data from. Now we only have jpeg. Basically, we didn't ask for an hikvision camera, it was they (chinese supplier) who chose that. But they also don't know much about the camera itself. This is a very complicated situation...

 

[sophiecentaur]

We asked a camera that we could extract raw data from. Now we only have jpeg.

I have already pointed out that, at your lev el of engineering, Raw or JPEG distinctions are not relevant.

Ideally, we would have a thermographic camera

IS that really what you want? do you have examples of thermographic cameras that cover 600C?

The only workaround I can suggest would be to recalibrate system, using the pixel values and their relative values over a range of out of range hot objects.

we don't have time

Comiserations. That's not uncommon when dealing with 'distant' suppliers. These systems call for an appropriate level of knowledge and expertise for which one has to pay.

 

[pcarvalho]

I have already pointed out that, at your lev el of engineering, Raw or JPEG distinctions are not relevant.

IS that really what you want? do you have examples of thermographic cameras that cover 600C?

The only workaround I can suggest would be to recalibrate system, using the pixel values and their relative values over a range of out of range hot objects.


Comiserations. That's not uncommon when dealing with 'distant' suppliers. These systems call for an appropriate level of knowledge and expertise for which one has to pay.

I forgot to mention one thing. We actually had a very similar system with a thermographic camera "http://www.coxcamera.com/en/bbs/board.php?bo_table=pro02_en&wr_id=2". This was bought from another supplier. As you can see they have an "extreme" temperatures measurement range. We don't need that "extreme temperatures", but we know that they used raw data to detect hot spots. I mean, we know that for each pixel, they had distance and raw data (adc value) and with their own algorithm they could spot hot spots at very long distances (thresholding). We want to do something similar. We asked for a raw camera from the chinese supplier and they said "yes the camera has raw". But that is a lie.

 

[pcarvalho]

Maybe I should have said this before.

 

[sophiecentaur]

We actually had a very similar system with a thermographic camera "http://www.coxcamera.com/en/bbs/board.php?bo_table=pro02_en&wr_id=2".

Maybe I should have said this before.

Yes. It would have saved a lot of wasted time.
Basically you have the totally wrong device and you seem surprised that it doesn't do the job you wanted. I don't see that PF can help you at all with your equipment sourcing problems. If you had actually identified a need for a radiometric camera and were prepared to pay for it then It may have dealt with hotter sources but you can hardly complain about that. Did you think that you could put together a low cost system for your customer. But there's not such thing as a free lunch.

I have to repeat my comments about the JPEG issue. Do you actually know what JPEG does to a RAW image? I suggest you read about it and what it's used for. It is a compression method for visible colour images (including moving) which start with three colour signals. You have a monochrome image and the system produces a false colour image for basically only cosmetic purposes. If you want to process the data more then you can easily get a Luminance signal with a bit of alternative processingl. Your data has only one channel. The output display is basically very crude and low res because that's all that's necessary. JPEG makes a signal which is easily copied and displayed.

We want to make our own thermal analysis app.

You will need to know a lot about suitable colour coding and file structures if you want to make the most of the available data. Maybe that's were you skills lie.

 

[tech99]

I suspect the camera has an automatic exposure adjustment, so that the dynamic range can be accommodated, and which will prevent reliable calibration.

 

[sophiecentaur]

I suspect the camera has an automatic exposure adjustment, so that the dynamic range can be accommodated, and which will prevent reliable calibration.

The thermal image that users want will be presented like a normal photo so auto exposure would be what you'd expect to see (plenty of contrast between hottest and coldest parts of the picture). The problem is that, for temperature measurement, the absolute energy flux per pixel has to be known (using Stefan's law). That either has to be done by an expensive, calibrated unit or with a reference area somewhere in every scene with a known temperature. A high quality instrument also needs to compensate for the actual size of the object being measured because of what would be crosstalk between adjacent pixels - sophisticated and costly probably.

The OP either didn't know this or didn't choose the right camera. I was a bit of a train crash from the start and the supplied camera should have been rejected as soon as it arrived. I think that the project was probably too hard for the OP, who (as we all have been) was over optimistic about the task. There could be a way back by using a reference hot object in every scene.

The data processing would be comparatively trivial to produce a 'fair' measurement.

Maybe this should be asked directly to a professor?

Who could charge as much as a proper radiometric camera. What's needed is really a design consultant, rather than a "professor".

TL;DR: We need to detect hot spots with a thermal (non-radiometric) camera. The camera retrieves JPEG.
We need to know if it makes sense to do the processing on top of JPEG.

what would be the maximum distance that the algorithm would work fine?

If you read around the topic you will see that distance is not included in Stefan's law. That's not intuitive but you should be aware of it.

 

[pcarvalho]

The thermal image that users want will be presented like a normal photo so auto exposure would be what you'd expect to see (plenty of contrast between hottest and coldest parts of the picture). The problem is that, for temperature measurement, the absolute energy flux per pixel has to be known (using Stefan's law). That either has to be done by an expensive, calibrated unit or with a reference area somewhere in every scene with a known temperature. A high quality instrument also needs to compensate for the actual size of the object being measured because of what would be crosstalk between adjacent pixels - sophisticated and costly probably.

The OP either didn't know this or didn't choose the right camera. I was a bit of a train crash from the start and the supplied camera should have been rejected as soon as it arrived. I think that the project was probably too hard for the OP, who (as we all have been) was over optimistic about the task. There could be a way back by using a reference hot object in every scene.

The data processing would be comparatively trivial to produce a 'fair' measurement.

Who could charge as much as a proper radiometric camera. What's needed is really a design consultant, rather than a "professor".

If you read around the topic you will see that distance is not included in Stefan's law. That's not intuitive but you should be aware of it.

To be honest It wasn't me who decided the camera. This is a complicated situation as I'm just an employee and I don't have decision making authority. I just gave the technical requirements and my boss sent them to the chinese supplier. The chinese supplier chose the camera. The system was handed to me so I could create an algorithm, very similar to the one mentioned above, to detect hot spots. We were told by the supplier that we could extract raw data from it, but that is not the case. So we tried other approach, by using jpeg images. That is why I created this topic. I just wanted to understand if I could do anything with this format. But we know that it is not possible to do anything unless we disable the auto gain control. We only knew yesterday that this option is hidden in the main page of the camera. We can't disable it. In conclusion we need a miracle now.
Hopefully this is more clear.

 

[sophiecentaur]

To be honest It wasn't me who decided the camera.

I am not apportioning any blame, I am just pointing out what is needed for your project to work. If you take no notice of what PF gives you then you waste your time with this enquiry.

. I just gave the technical requirements

Are you sure your specification was correct? What did you tell him was needed?

We were told by the supplier that we could extract raw data from it, but that is not the case.

I ask again why you want to do that? What advantage do you get from that? It is a red herring for a simple system like yours. A JPEG file will not have lost relevant information.
Any software that 'gets back' to a RAW signal cannot re-insert perfectly the lost data that JPEG has already eliminated (JPEG is not a lossless algorithm)
Read (or ask your boss) what it so special about RAW at this stage? AFAIK, JPEG does no exposure control. Do you know what JPEG will do to enhance / change your data significantly? What did your JPEG images not tell you? They will just bit-rate reduce the RAW data. You haven't posted any images but they will not be High Res. In the case of your camera you will get an image with no reference temperature so you can introduce your own - for instance a black can of boiling water somewhere. The pixel value will give the value of the temperature of 100C. (ignoring albedo). If you find a way to stop the agc then you will still need a reference each time. But can you rely on the linearity of your system if 600C if it causes clipping and 'squashing' (cross modulation of signals)? Cameras with extreme settings may not just reduce the gain; you can't tell.

There is a problem with the radiant flux / temperature calculation in that the temperature ratio of two objects is given by the fourth root of the ratio of the radiant flux. That can mean that the accuracy of temperature may be far worse than the accuracy of radiant flux measurement