Comp. Neuro. introductory textbooks

[nobahar]

Hello!

I know some people on the forum are computational neuroscientists (or related fields), and I was wondering if anyone could recommend a good introductory book. One that goes step-by-step through how the equations come about and why they are the way they are. Rather than books that simply give the equations and say "...this models the neuron...", etc.
I'm guessing textbooks might be stronger on some topics than others. I found one that has a good introduction to the fundamentals of information theory quite well: the reasoning behind the equations, proofs, and so on; but it doesn't give a very good explanation of how neurons are modeled for some reason, and just tends to give the equations.

Anyone have any books they found particularly useful?

 

[madness]

The standard textbook:

http://www.amazon.com/dp/0262041995/?tag=pfamazon01-20

A good textbook on information theory applied to neuroscience:

http://www.amazon.com/dp/0262681080/?tag=pfamazon01-20

Dynamical systems applied to neuroscience:

http://www.amazon.com/dp/0262514206/?tag=pfamazon01-20

I would say the first one is best for understanding how neurons are modeled biophysically. Perhaps this one would be even better for this but I haven't personally read it:

http://www.amazon.com/dp/0521877954/?tag=pfamazon01-20

 

[atyy]

https://www.amazon.com/dp/0262231840/?tag=pfamazon01-20

http://www.sciencemag.org/content/326/5951/379.summary

 

[Pythagorean]

I am part of the Facebook 'Theoretical Neuroscience' group (including several instructors and well-knowns in the field like Bard Ermentrout and Peter Erdi) and they had this discussion some time ago. The general consensus was that Dayan & Abbott's Theoretical Neuroscience was okay, but all bottom-up (no top down approaches).

Neuroscience: A Mathematical Primer, by Alwyn Scott was mentioned, but not much appreciated by faculty, Dynamical Systems in Neuroscience by Eugene Izhikevich is one of my favorites, but again, to idiosyncratic for faculty teaching the course.

Ermentrout suggested Hugh Wilson's Spikes, Decisions, and Actions: The Dynamical Foundations of Neuroscience as the best book, but also mentioned his own book, Mathematical Foundations in Neuroscience.

 

[Pythagorean]

Here is Hugh Wilson's Book, free to download from Wilson himself (as Ermentrout had mentioned). It's being sold used for $1000 in hard cover (because it's out of print)! It's received excellent reviews at Amazon:

http://cvr.yorku.ca/webpages/wilson.htm#book

 

[nobahar]

Thanks for the suggestions. Some I can get hold of, some I can't. Thanks for the free online book Py; I had a brief look and it seems to take you through step-by-step, starting from 'simple' differentials. I also looked at Theoretical Neuroscience by Dayan, suggested by madness.
Differential equations, eigenvalues, eigenvectors... sheesh...

Anyway, thanks again for the recommendations.

 

[Pythagorean]

I think what I like about that free book is that it has the word "decisions" in the title, hinting that it might actually cover top-down models as well as the standard bottom-up models. I downloaded it, but haven't had the chance to flip through it yet.

 

[nobahar]

I think what I like about that free book is that it has the word "decisions" in the title, hinting that it might actually cover top-down models as well as the standard bottom-up models. I downloaded it, but haven't had the chance to flip through it yet.

Hi py, what do you mean by "bottom-up" and "top-down" models? I'm new to this (obviously). I have to keep setting it aside to tackle the kind-of 'pre-requisites', like electrostatic interactions and circuits, and the maths.

 

[Pythagorean]

Say you have some behavior you observe that you want to model. Say that when you poke a creature, 70% of the time, it retracts a protruding appendage, but 30% of the time it doesn't.

There's two ways you can model this. The simple way would be to model it as a black box with an input and an output. The input would be the poke, and the output would be the retraction (or lack of retraction). You could simply model the black box to generate a random number between 1 and 0, with all numbers on that interval being equally probable. Then if the number is between 0 and .3, the output is no retraction, but if the number between .3 and 1, there is a retraction. This would be an example of a top-down model. You're modeling exactly what you observe, nothing more, and you're not asking about the mechanisms underlying it. This is maybe even too pure of an example of top-down modeling, since it doesn't really allow for you to learn anything about the system itself, but it can be used as a component in a bigger system.

A more complicated way to model it would be to model the neurons as differential equations with parameters tuned to the right values. This has the advantage of being able to talk about mechanisms, but it has the disadvantage that you may never find the regime of parameter sets and inputs that reproduces the behavior you observe because there's so much complexity. You may even be missing an important part of the story and end up forcing the neurons out of their natural paramter ranges to produce something like the behavior, when all along, you were missing another member or property of the ensemble (like astrocytes, or another bridging neuron, or a specific synapse).

Generally, the bottom-up model is preferred in the physical sciences, but if your system gets to complex with lots of interacting parts, it might be better to use a top-down model. Ultimately, the most complicated stuff uses both: you start with a top-down model of all the interacting black-boxes, but as you gain understanding, you replace the black boxes with components that are more bottom-up modeled.

 

[atyy]

There's two ways you can model this. The simple way would be to model it as a black box with an input and an output. The input would be the poke, and the output would be the retraction (or lack of retraction). You could simply model the black box to generate a random number between 1 and 0, with all numbers on that interval being equally probable. Then if the number is between 0 and .3, the output is no retraction, but if the number between .3 and 1, there is a retraction. This would be an example of a top-down model. You're modeling exactly what you observe, nothing more, and you're not asking about the mechanisms underlying it. This is maybe even too pure of an example of top-down modeling, since it doesn't really allow for you to learn anything about the system itself, but it can be used as a component in a bigger system.

Is that really a "model"? If I just put in what I observe, then it's just the data, which I guess is a "model". But it doesn't seem to make a prediction, since the input space is restricted exactly to stimuli already used in the experiments.

Do you happen to have Volterra series or an equivalent (for all practical purposes) representation in mind?

 

[Pythagorean]

You're right; I thought so too, which is why I put my last sentence. I went too much to the extreme of exactly modeling observations and no more.

Volterra-Lotka is a good example:

http://en.wikipedia.org/wiki/Lotka–Volterra_equation

We don't couple a bunch of bunnies and foxes together we just model the general population of each as a function of both the population of the other and the organisms own population.

 

[atyy]

You're right; I thought so too, which is why I put my last sentence. I went too much to the extreme of exactly modeling observations and no more.

Volterra-Lotka is a good example:

http://en.wikipedia.org/wiki/Lotka–Volterra_equation

We don't couple a bunch of bunnies and foxes together we just model the general population of each as a function of both the population of the other and the organisms own population.

I see. Let's see if I understand your terminology: an integrate and fire neuron would be both "bottom up" with respect to say decision making, and "top down" with respect to the Hodgkin-Huxley models, since eg. m,h,n are not explicitly modeled, and in fact will in general be less faithful to reality than the HH equations, but that error may not matter for one's purposes.

 

[Pythagorean]

With respect to decision making, I'd think a integrate and fire neuron would be bottom-up still because it says nothing about observations with respect to decision making. It's still a matter of putting integrate and fire neurons together and hoping some thing like decision making comes out of it.

You'd actually have to model the decision of a person as the particle in state space, rather than the state of the neuron, I'd think. So the VL models the statespace of the two populations as one particle; it doesn't take a bunch of rabbits and bunnies and couple them together and hope for the best.

 

[atyy]

With respect to decision making, I'd think a integrate and fire neuron would be bottom-up still because it says nothing about observations with respect to decision making. It's still a matter of putting integrate and fire neurons together and hoping some thing like decision making comes out of it.

You'd actually have to model the decision of a person as the particle in state space, rather than the state of the neuron, I'd think. So the VL models the statespace of the two populations as one particle; it doesn't take a bunch of rabbits and bunnies and couple them together and hope for the best.

Yes, that's what I meant to convey. Top-down would be like http://books.google.com/books?id=CM5lWq2zFbcC&source=gbs_navlinks_s :)

 

[Pythagorean]

Yes, that's what I meant to convey. Top-down would be like http://books.google.com/books?id=CM5lWq2zFbcC&source=gbs_navlinks_s :)

Ah yes, sorry. I reread your post. Yeah, that's consistent with my terminology. Will check out the book once I'm at a computer.

update: yeah, that fits the the stereotype I've developed that psychologists are more likely to utilize top-down models, neuroscientists bottom-up models. Nonlinear scientists both :)

 

[madness]

There is a bit of a problem with the definition of top-down and bottom-up in neuroscience, in my opinion. In the brain, top-down can refer to the modulation of low-level brain areas by high level brain areas, e.g. the influence of attention on perception. There is also the concept that a complex system can have emergent behaviour which influences its constituent components, similar to how society shapes an individual. These are both different from what you have discussed here - there are multiple uses of the term.

 

[Pythagorean]

Yes, we're taking about modeling theory specifically here, not the brain processes themselves.

 

[madness]

The idea that emergent behaviour influences constituent components is a modelling issue. For example, Nunez models the neocortex in terms of neurons embedded in global synaptic fields (http://plaza.ufl.edu/johncad/nunez.pdf). A bottom-up model would have no need to explicitly introduce a top-down influence, it would simply emerge through the low-level dynamics.

 

[Pythagorean]

The idea that emergent behaviour influences constituent components is a modelling issue. For example, Nunez models the neocortex in terms of neurons embedded in global synaptic fields (http://plaza.ufl.edu/johncad/nunez.pdf). A bottom-up model would have no need to explicitly introduce a top-down influence, it would simply emerge through the low-level dynamics.

I agree, but this is essentially the same as any other system. We accept only weak emergence so far. That is, even top-down influences can be explained in terms of micro-properties (like wave properties that drive water molecule trajectories can be explained by the coupling between water molecules and the statistical group behavior of the ensemble).

A lot of problems come down to the "levels fallacy". People have this belief that, for instance, life is explained completely by cells, cells completely by molecules, molecules completely by subatomic particles. But this isn't exactly true. A life form has more than cells: it has digestive juices, connective tissues, and stimulus from environment (or other organisms) on it. So the levels aren't perfectly isolated, each contributing to the next. They're mixed up in a complicated hierarchy. At some point, you have to be careful to acknowledge that the line you draw between one level and the next may be arbitrary.

 

[nobahar]

This conversation has gotten away from me a bit.
In an abstract, very generalised sense, are top-down and bottom-up approaches somewhat analogous to the following?:
Using the equation you posted previously, in a bottom-up approach you would speculate that a prey population would increase exponentially in the absence of predation, so there is a term to account for that. However, the population is also dependent on the number of predators, and the 'rate of predation' is dependent on the number of predators, but also the number of prey, and so there is a term to account for that aswell, and so on until equations are built to model the system. This is building an equation from some core assumptions, and then the equation(s) is(are) used to predict future outcomes given certain inputs (e.g. starting values).
A top-down approach on the other hand, would measure the population numbers at different points, look for patterns, and then come up with an equation that matches the pattern, and use this to make future predictions given some input values. In this case, the equation is built from trying to match the data, and isn't based on some 'fundamental' picture of what is happening: it isn't built from an understanding of underlying mechanisms, although it might give some insight into what they are. Using the prey/predator example, the equation might come from attempting to match the data, and then by looking at idealised examples (e.g. no predator) it might be possible to 'break-up' the equation to see what it's doing, and one could deduce that one of the equations used to model the system suggests that the prey population would increase exponentially in the absence of predation, whereas this might not have been obvious before.
Hopefully that makes sense...
Any response appreciated.

 

[apeiron]

We accept only weak emergence so far. That is, even top-down influences can be explained in terms of micro-properties (like wave properties that drive water molecule trajectories can be explained by the coupling between water molecules and the statistical group behavior of the ensemble).

Not sure who this "we" is. As Madness looks to be arguing, a systems science or hierarchy theory approach says "more is different" because it allows for the fact that macroscale top-down causality shapes up the very microscale bottom-up elements that constitute the whole.

So this is where the difficulty lies. With neuron firing, for instance, global attentive states bear down to shape the individual receptive fields. The global state may be constituted of this local firing, but at the same time, the local firing is shaped by the global state.

What models of this strongly emergent behaviour need to do is model the local elements as some set of degrees of freedom that can then be subject to emergent constraints. So the local elements are themselves dynamical (in ways the model accurately captures).

This would be the approach for instance of Grossberg's ART models, or Friston's Bayseian Brain.

A snapshot of a system at any point in time will indeed make it appear that the definite actions of the microscale are all that are causing the macroscale state. The causality is all bottom-up. But this is an artifact of taking such a restricted view. Systems and processes live in time, and holistic models would attempt to capture all the relevant spatiotemporal scales of action.

 

[Pythagorean]

Not sure who this "we" is. As Madness looks to be arguing, a systems science or hierarchy theory approach says "more is different" because it allows for the fact that macroscale top-down causality shapes up the very microscale bottom-up elements that constitute the whole.

So this is where the difficulty lies. With neuron firing, for instance, global attentive states bear down to shape the individual receptive fields. The global state may be constituted of this local firing, but at the same time, the local firing is shaped by the global state.

What models of this strongly emergent behaviour need to do is model the local elements as some set of degrees of freedom that can then be subject to emergent constraints. So the local elements are themselves dynamical (in ways the model accurately captures).

This would be the approach for instance of Grossberg's ART models, or Friston's Bayseian Brain.

A snapshot of a system at any point in time will indeed make it appear that the definite actions of the microscale are all that are causing the macroscale state. The causality is all bottom-up. But this is an artifact of taking such a restricted view. Systems and processes live in time, and holistic models would attempt to capture all the relevant spatiotemporal scales of action.

One of the many representatives of we:

Paul Humphreys and Cyrille Imbert (eds.), Models, Simulations, and Representations, Routledge, 2012, ISBN 9780415891967.

And what you describe (in general) is anyway, under the scope of weak emergence (see the definitions from the above). The only difference between weak and strong emergence is strong emergence has extra philosophical baggage. Scientifically, there is no significance to calling your boundary conditions "causing". It is a matter of extensive vs. intensive properties that constitute emergence. A purely extensive system behaves only as a sum of its parts, but boundary conditions in spatiotemporal systems can have intensive properties and group behavior emerges that you would not get from a single member of the ensemble. There is no doubt that the boundary conditions (including the coupling term) affect the group behavior; they are everything about emergence for many network systems in nature.

As we are now, Madness was describing the model itself (top down vs. bottom up) and talking about the processes being modeled as is typical in Cognitive Sciences. That's independent of whether you use a top down vs. bottom up approach to the modeling.

i.e. you could model a system that displays both top-down and bottom-up processing using both top-down and bottom-up modeling methods. Which is why I conceded to Madness's statement that top-down bottom-up mean different things in different contexts, but I also noted that we were talking only about the modeling approach itself, not the thing being modeled, and so the context was already set.

But yes, pain is the canonical example they make of top-down processing in neuroscience classes (by being conscious of a wound, it can hurt more). Of course, it's not just top-down, it's more like bottom-up sensory being modulated by top-down focus. The easiest way to model this would be to use bottom-up processing to model the wound (a cut feels different than a bruise) and use focus/attention as a function (or weight) that modifies a term in the bottom-up process... i.e. multiply the pain based on how much attention focus is on it (and this would be the top-down portion of modeling).

Of course, you could also model the whole system bottom up (even the top-down process). An example would be an integrated circuit that receives three inputs. One at the eyes, and one at the skin and one from the memory banks. And the input from the eyes and the input from the skin would have to match an association process through the memory banks and produce an integrated response. This particular example is currently impossible as far as I know (too much and too little information at the same time) ... so the top-down modeling approach is often convenient for top-down considerations which makes it easy to conflate that top-down process = top-down modeling approach.