Can a Body Language Algorithm Reveal Our True Emotions?

[w4y021]

TL;DR Summary

brain storming

These days there is a question confuses me if it can be real or not.
2-3 years ago when I was looking around papers about computer sciences on the web, I came across to a sceintific paper which mentions an algorithm about analysing your language even it's a foreign language. So when you speak behind a camera, the algorithm reads the words from your lips movements and converts them into subtitles. Of course I don't know if it gives a very high percentage of precision level or not. But at those years the research was new (I think) so there will be some faults or mistakes and after algorithm will be updated of course.

This paper turned some light of ideas in my head: :)
For instance if we can convert lip movements of a person to subtitles (namely words) with revealing the language which the person speaks, then can we do a body language algorithm that reveals the emotional and mental conditions. Of course there are lots of parameters in this algorithm. Because it consists of many titles like the psycological situation of person. For example think about the one gesture which is a result of different emotions. It'll be a very complex problem. So this idea is like a lie detector a little :] But if it can be real or not I really don't know.

 

[Klystron]

The OP might be interested in the science of kinesiology. Though more concerned with body movement than body language, the science is quite real and effective for physical therapy and improving performance of dancers, athletes and people with physical disabilities.

Kinesiology is the study of human and nonhuman animal-body movements, performance, and function by applying the sciences of biomechanics, anatomy, physiology, psychology, and neuroscience. Applications of kinesiology in human-health include physical education teacher, rehabilitation, health and safety, health promotion, workplaces, sport and exercise industries.

 

[Helios]

Now this would be interesting for a poker playing computer, that tries to determine if the opponent is bluffing or has a winning hand. Along with body language would be breathing pattern, body temperature, perspiration, etc.

 

[w4y021]

Now this would be interesting for a poker playing computer, that tries to determine if the opponent is bluffing or has a winning hand. Along with body language would be breathing pattern, body temperature, perspiration, etc.

Good idea :] Especially I like the way of collecting the information about body temperature, breathing patterns etc.

 

[anorlunda]

Now this would be interesting for a poker playing computer, that tries to determine if the opponent is bluffing or has a winning hand. Along with body language would be breathing pattern, body temperature, perspiration, etc.

In poker, they call that a "tell". So your machine would be a Teller. I would name it "Edward".

 

[f95toli]

There are companies that sell product based on this idea (I believe Kepler is one example, China is rumoured to be using it for its general population). They are indeed uses as "lie detectors" but the underlying science is very, very dodgy.
The obvious problem (just as with regular lie detectors) is that the whole premise of an "unconscious" response which depends on your state of mind and is consistent enough that it can be used in any meaningful way is most likely flawed. How we react depends on so many different factors that it becomes meaningless.

 

[Klystron]

There are companies that sell product based on this idea (I believe Kepler is one example, China is rumoured to be using it for its general population). They are indeed uses as "lie detectors" but the underlying science is very, very dodgy.
The obvious problem (just as with regular lie detectors) is that the whole premise of an "unconscious" response which depends on your state of mind and is consistent enough that it can be used in any meaningful way is most likely flawed. How we react depends on so many different factors that it becomes meaningless.

This is why I recommend using kinesiology as one basis for your algorithm. Most "lie detectors" I have studied are based on wishful thinking with dubious results. I often volunteered as a human subject while working at various Silicon Valley 'think tanks'. One experiment taught us to control autonomic body responses to spoof so-called lie detectors with a few minutes training.

As a young programmer I helped a group of student "movement scientists" with a project that included reading "tells" in poker and bridge players. One of my fellow students used films from bridge tournaments to detect player collusion, signalling and other cheating based on movement. Results were published but I have not read peer reviewed papers.

The press likes expressions such as 'body language' but I think scientists use 'bio-mechanics' to distinguish from linguistics including voice analysis for micro-tremors and other verbal "tells". Years later I worked in a speech technology lab and found many similarities with movement science. Spoken language and body movements including eye motions combine in human communication.

Collaborative projects included analyzing films of politicians under stress for body motions particularly face, head and shoulders coupled with voice decomposition to detect tell-tale stress markers that might indicate falsehood. Richard Nixon was a favorite subject with a long career under camera but minimal professional training. Ronald Reagan as a trained actor had more control of movement and voice but one can find publications asserting evidence of cognitive decline in later recordings.

So, in my limited experience as a software engineer there are useful algorithms based on science but also lots of malarkey written about lie detectors. Suggestion: poker tournaments are filmed, recorded and broadcast. One could use these to train an expert system then compare your results with actual winners and losers.

 

[anorlunda]

If you really want to become richer than Jeff Bezos, make a machine that can read the "tells" of young people's attraction to other young people. It could outsell acne medicine.