Image Processing Part 1: Skin Temperature From Thermal Images

12.19.2017

The senior design project I have been working on in school is a "smart mirror". Equipped with a thermal and RGB camera, its primary purpose is to analyze a user's face and give health feedback. The subsystem I was responsible for was image processing and my goal was to analyze the images to obtain health metrics. Here I go over getting a skin tempeprature metric from thermal images.

fever
Ever wanted your mirror to tell you if you're catching a fever?

Skin Temperature

Skin temperature is a health metric we care about because it acts as an approximation of internal body temperature. Changes in body temperature may be a health concern such as a fever so it seems important to measure and track.

Process

flow diagram
Diagram for the process of getting skin temperature.

The thermal images have dimensions of 80x60 pixels and come from a FLiR Lepton. Each pixel has a value and the higher that value the hotter the temperature had been measured. From these images I wanted to extract a person's skin temperature value.

Altough it may be intuitive to just take the highest value as the skin temperature, it is not guaranteed that a person will be the warmest thing in the image. There may be a light bulb, a candle, or a cup of coffee in the background. Therefore some simple image processing techniques are needed to remove these "hot spots".

The first step is to find the median of the thermal image and use it as the value for a threshold. There are several other threshold values to choose from e.g. the average pixel value or using Otsu's method. I use the median because it is less affected by very hot or cold objects caught in the image. Any pixel with a value over that value will be white and any under will be black. Here I refer to the white part as the "mask".

skin temperature process images
Visualized Process: Original, Thresholded, Open+Close, and Orginal Masked

The mask needs to be 'adjusted', so the second step is to apply some morphological transformations. The first is called an open and it will remove smaller white areas caused by warm background objects. The second transformation is called a close and it does exactley what it sounds like; any holes in the mask caused by something like glasses or thick beards will be closed up.

Now the mask approximately covers the person in the image. The final step is to use this to "mask" the original image and only look for the highest pixel value in that area. This highest value will then be our skin temperature metric and the process is complete.

Corner Cases

My assumption is that the there is one person in the image and that they are the prominent figure in the image. Any object hotter than skin that is in front or on the edge of that person will not be removed from the mask and cause an error. The person should also consistently either wear or not-wear glasses. This is because the glass blocks one of the warmest areas of the face from the view of the thermal camera, and so one reading without glasses may be higher than without.

Github, for OpenCV code, in C++.