Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Video image processing

  1. Oct 20, 2015 #1
    Is there a computer algorithm that can determine if a person it moving their right arm or there feet, or if they are running or not ?
  2. jcsd
  3. Oct 20, 2015 #2


    User Avatar
    Gold Member

    I guess there are programs that can determine it ( not algorithms ).

    The only thing a camera/computer can see is an amount of pixels with different colors within a frame. You can make an algoritm that recognizes some patterns in the image, like an edge: Scanning the image horizontally and finding a sudden color change along a vertical line, it has found a vertical edge. In this way you can convert the image into a cartoon:
    There are a lot of ways to recognize a figure as a person. One way is to Fourier transform the outline of the figure, regarding the edges as a series of complex coordinates. The Fourier transform is a "finger print" of a person. Adding a tail to the right figure, the Fourier transform will change and become a finger print of a dog. The "smart" charasteristic is, that if you cancel the 1. harmonic of the right figure, you can rotate him and let him crawl uphill or downhill: He will still be recognized as a person because the angle doesn't matter when the 1. harmonic is missing.

    So the idea is to sketch a lot of persons/dogs in various positions, make "finger prints" and tell the computer: this is a person and this is a dog. When the computer has such a look-up-table of finger prints and sees a person or dog, it can distinguish a dog/person by closest match.

    To distisguish left/right arm is more complicated: Am I seeing the front or the back of a person?

    Attached Files:

  4. Oct 20, 2015 #3
    This is the type of things that neural networks will be good at. Computers are not great at this yet, there is a joke in computer science that highlights the differences between what computers and the brain are good at.

    Boss: I need a program that takes the entire literary history of humanity, and maps out the use of pronouns throughout history.
    Engineer: Sure, give me twenty minutes.

    Boss: I need a program that tells me if this picture is of a dog.
    Engineer: Sure, give me five years and a dozen research assistants.
  5. Oct 20, 2015 #4
  6. Oct 21, 2015 #5


    User Avatar
    Gold Member

    Well, reducing 6 million pixels as inputs in a neural network, to say 200 harmonics will make it easier for the network to make a decision.

    Often neural networks are trained by the "steepest chase" method. How will a neural network by itself find a steepest chase?
    You are the one to configure groups of neurons and the network between them. Where will you start amongst 6 million pixels?
  7. Oct 22, 2015 #6
    Yes, such algorithms exist and are present in commercial products -- such as Microsoft's XBox and Kinnect system. Note that neural networks can be used, but so are various forms of decision trees with Harr-like features and other techniques. You may want to find a general text on Computer Vision / Machine Vision as it is a very broad and active area of research and there are open source libraries like OpenCV that may interest you. There are even javascript implementations of some algorithms like face detection -- see trackingjs.com
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook