Retina pre-processing - what does it look like

[sazr]

Do we know what sort of preprocessing the retina does? And more importantly what the output of this preprocessing would look like? I'm interested from the perspective of computer vision uses, I can copy what the retina is doing to efficiently pre-process (and remove redundant information) and perform higher up processing that the brain would do.

If anyone could describe visually what the output of retinal pre-processing would look like that would be extremely helpful. For example, would an example output look like just edges (so black wheres there's no edges and white/colour? where there are edges). I am interested in any/all eyes that have retina's not just humans.

 

[BillTre]

This is a complex question.
In vertebrates, there are many different kinds of different retinal ganglion cells that convey different kinds of data to the brain. Retinal ganglion cells are the output cells of the retina. What they convey is the post-retina-processed information.
Not all of them go to the same place. Most project retinotopically from the retina to the optic tectum (or superior colliculus; same thing different name in different species). These cells are involved in information about where things are in space around the animal. The retina can be considered a flat sheet of cells, each receiving light from different parts of the visual field, so do cells from each part of the retina project to the optic tectum in manner that maintains the topology of the visual field. Thus, the population of cells at each step a long this visual pathway maintains the spatial relationships of the parts of the visual field.
Here is an actual look at visually evoked activity in the optic tectum of a larval zebrafish as shown by calcium sensitive, light emitting reporter protein in tectal cells as it watches a prey item (a large paramecium) zip around (in speeded up time). In this, you can see the paramecia and the optic tectum, but not the rest of the immobilized fish.

I am guessing this is the kind of thing you are most interested in.

Some other retinal outputs project to other areas of the brain (such as pineal or SupraChiasmatic Nucleus (SCN) to convey general illumination information to the circadian rhythm parts of the brain. The spatial origin of the visual inputs are not so important in this case and the spatial relationships with th evisual field are probably discarded.

Here is a (probably incomplete) list of vertebrate retinal ganglion cells from wikipedia with some of their properties. I am also thinking lists like this are what you are looking. I have seen more extensive ones, but they are probably tucked away in specialist (retinal or visual science) literature. If you can get into a university research library where a department has a bunch iof vision researchers you could probably find many books and journals on this.
There are also central vs. peripheral differences in the output cells (more or certain kinds centrally of peripherally) and some animals have differences between areas of the retina that will be looking at the sky (predators?) vs. other areas. An extreme case of this could probably be found in the Anableps (four eyed) fish, which swims around in shallow areas with the top halves of its eyes out of the water, looking around through the air, while the bottom halves of its eyes are looking through the water. The different top and bottom areas of these eyes would be expected to have different output cells looking at different properties, but I could not find any studies on this.

Going further afield into non-standard biology, compound eyes found most commonly in arthropods have a completely different structure where much more processing occurs downstream (as the neural impulse flows). The photoreceptors of the compound eye project a small distance to the flies optic lobe (a part of the fly brain) that directly underlies the compound eye, with no intervening neurons. Most of the signal processing starts there.
There are other kinds of eyes in arthropods (usually called ocelli). They probably have specific functions with specific cells doing their own kind of processing steps. These are not so much worked on.

 

[atyy]

Two important operations are
- sign inversion (dark causes excitation, light causes inhibition in off center cells - this is important for complex cells in the visual cortex, which are phase invariant oriented edge detectors)
- elements of edge detection (center-surround antagonism)
http://www.cns.nyu.edu/~david/courses/perception/lecturenotes/ganglion/ganglion.html
Early efforts at using biology to inspire artificial neural networks for vision are:
https://en.wikipedia.org/wiki/Neocognitron
http://cbcl.mit.edu/publications/ps/serre-wolf-poggio-PAMI-07.pdf
Nowadays, with deep learning so powerful, there is not so much need to hand engineer the early stages of artificial neural networks for image recognition. However, one thing that remains is that each retinal ganglion cell does the "same" operation, just translated in the visual field - this symmetry is implemented by "convolutional neural networks".
https://en.wikipedia.org/wiki/AlexNet
https://papers.nips.cc/paper/4824-i...n-with-deep-convolutional-neural-networks.pdf