Artificial neural network(s) and Computational neuroscience

[Eagle9]

Hello guys

I need your help in order to clarify the following issue

I am PhD student in neuroscience and I am dealing with brain’s neuroanatomy and physiology in case of Autism. But I am more interested in Computational neuroscience, Artificial neural network(s), the problem of consciousness and things like them. To my mind, this is the most complex, a bit mysterious and intriguing topic to be explored. I want to study these sciences.

But I do not exactly know what is the essential difference between Computational neuroscience and Artificial neural network(s). Do they explore the same things? Or similar things? Do they (at least partly) overlap each other? Which one is more close to the problem of consciousness? Do they equally need the knowledge in Mathematics? Cybernetics? Which of them should I choose (University/Faculty/Program)?

 

[jedishrfu]

I would say they are very different topics. ANN's are used in problems where you don't know an algorithm to make a decision and so you use lots of data to train the ANN to discriminate between conditions and select the best appropriate answer.

Watch this sequence of videos to understand ANN usage and how they work their magic:



In contrast computational neuroscience seeks to mimic how the brain actually works.

You could liken the two approaches to astrology vs astronomy where ANNs arrive at their conclusion being like astrology using anecdotal data to arrive at a conclusion (which we don't know how it came up with its conclusion) whereas computational neuroscience is more like astronomy using what we know about the brain's engineering and function to accurately model how the brain works.

ANN's sometimes give the wrong answer and when that happens you have to go back and train it some more. Sometimes, you might overtrain the ANN but this is just part of how you tune ANNs for problem you're trying to solve.

 

[Eagle9]

Watch this sequence of videos to understand ANN usage and how they work their magic:

I watched the first video, so it is perceptron, right? I am aware about its history and usage. So, if I understood correctly the Artificial neural network(s) is/are based on this concept?

ANN's are used in problems where you don't know an algorithm to make a decision

And the solution is found by the method of “Trial and error”?

In contrast computational neuroscience seeks to mimic how the brain actually works.

And therefore in order to understand how the (natural/human) brain works the Computational neuroscience is needed and not Artificial neural network(s), right?

You could liken the two approaches to astrology vs astronomy where ANNs arrive at their conclusion being like astrology using anecdotal data to arrive at a conclusion (which we don't know how it came up with its conclusion) whereas computational neuroscience is more like astronomy using what we know about the brain's engineering and function to accurately model how the brain works.

So, please tell me what kind of knowledge in Mathematics/Cybernetics is needed for these two different fields

 

[jedishrfu]

For ANN programming, statistics, linear algebra, machine learning

For brain simulations you would need additionally physics, modeling of physical systems, differential equations

 

[Eagle9]

additionally physics, modeling of physical systems, differential equations

At a very profound level or at moderate one?

 

[atyy]

And therefore in order to understand how the (natural/human) brain works the Computational neuroscience is needed and not Artificial neural network(s), right?

An understanding of artificial neural networks is important for understanding biological brains. The main feature in which they diverge is how they learn. ANNs are trained by backpropagation, which is not biologically plausible.

However, another aspect of machine learning does show up in biology - the temporal difference error describes how dopamine neurons behave under some circumstances.

 

[Eagle9]

Please tell me which programming language is more suitable for Computational neuroscience? C++ or Python? Or maybe others?

 

[jedishrfu]

Machine learning and deep learning tend to use python or Matlab. However Julia is beginning to make inroads in these areas because of its speed of execution and ease of use. Julia is similar enough to Matlab that programmers of Matlab can move to it easier than those in python. It also has interoperability with python and R allowing fo mixed language solutions or leveraging legacy code in current development.

 

[Matterwave]

Hello guys

I need your help in order to clarify the following issue

I am PhD student in neuroscience and I am dealing with brain’s neuroanatomy and physiology in case of Autism. But I am more interested in Computational neuroscience, Artificial neural network(s), the problem of consciousness and things like them. To my mind, this is the most complex, a bit mysterious and intriguing topic to be explored. I want to study these sciences.

But I do not exactly know what is the essential difference between Computational neuroscience and Artificial neural network(s). Do they explore the same things? Or similar things? Do they (at least partly) overlap each other? Which one is more close to the problem of consciousness? Do they equally need the knowledge in Mathematics? Cybernetics? Which of them should I choose (University/Faculty/Program)?

I don't have too much experience with computational neuroscience, but I do know quite a bit about ANNs and other types of machine learning, so I guess I'll chime in there.

Artificial neural networks are modeled after the brain, but to be honest, it seems quite far-fetched to really call them a model of the brain. What they are are specific mathematical models for transforming given (numerical) inputs to some (numerical) outputs. These days, the popular neural networks are either the "feed forward" variety (multi-layer perceptrons or convolutional neural networks) or of the "recurrent" variety (Recurrent neural networks). These types of neural networks are often used to perform what is known as "supervised learning". I.e. you feed these neural networks a set of inputs and desired outputs and these neural networks learn (via back propagation as jedishrfu said) to replicate the outputs given inputs. The hope is that given enough input-output pairs, the network will learn to generalize and be able to learn appropriate outputs for a wide variety of inputs (inputs which aren't labeled with desired outputs before hand). An example of such a problem is image classification where if you feed a neural network (generally a convolutional one) a bunch of images (of cats, and dogs, say) and tell it the expected outputs (are they cats or dogs?) then given enough samples (of cat/dog picture to label pairs) hopefully the neural network will learn to classify images as cats or dogs without you providing the appropriate answer beforehand. Viewed from this perspective, these ANN's are really nothing more than fancy function approximators (some way to map a given input X to a given output Y: $f:X\rightarrow Y$.

There are other ways in which an artificial neural network can be made though and those might be of more academic interest. This includes the (2-way) Restricted Boltzmann Machines or the (any different way) Hopfield nets. These are "unsupervised learning" and they basically take some input and produce an output without you specifying a desired output to train it. They are not used nearly as much as the previously mentioned varieties of ANN's though.

The general gist though is that ANN's are a specific technique to model inputs and outputs. They are inspired by the brain, but don't necessarily correspond to actual brain functions.

Lastly, atyy mentioned in his post the temporal difference error - this is an error which shows up in reinforcement learning (the third type of machine learning - it's partially supervised and partially unsupervised) - specifically in the subclass of temporal difference learning algorithms (e.g. Q-learning or SARSA). It has nothing a priori to do with neural networks - it is a method to help assign rewards and punishments to reinforcement learning agents.

 

[Eagle9]

Machine learning and deep learning tend to use python or Matlab. However Julia is beginning to make inroads in these areas because of its speed of execution and ease of use. Julia is similar enough to Matlab that programmers of Matlab can move to it easier than those in python. It also has interoperability with python and R allowing fo mixed language solutions or leveraging legacy code in current development.

I am planning to learn Python (I am not absolutely aware about programming, but others advised me this program. They told me that it is relatively easy (much easier than C++) program, that it is widely used in science and etc.), will it be useful in Computational neuroscience (after reading the answers in this thread I am more interested in this field rather than in Artificial neural network(s))? I hope yes

Matterwave
Well, thanks for such big and thorough, detailed answer. One of the previous posts said:

In contrast computational neuroscience seeks to mimic how the brain actually works.

So, my interests shifted to Computational neuroscience, therefore I want to know what kind of knowledge should I have for this field. I am learning now Calculus and Physics (some basic one, for Biologists) and I want to learn at least one programming language, therefore I am finding out which language should I learn

 

[Matterwave]

Glad I could help. :)

I don't know much about computational neuroscience so I can't really provide too much insights there. I would imagine calculus, linear algebra and differential equations would be very important. As for programming languages, I think Python is a good starting choice. It is easy to get started in but it also has a huge community of open source programmers who have written a ton of libraries and packages to let you do essentially everything you could want to do. The general complaint I hear with regards to python is that since it's an interpreted language, it's going to be slower than a compiled one like C++. However, for heavy numerical computations, the developers at numpy have written a lot of optimized code to take care of all that kind of stuff. In fact, a lot of numpy's backend is written in C++ for pure speed considerations, but you are only exposed to the python API for convenience.

If you want to do python for scientific computing, I suggest getting the anaconda distribution - it comes with a ton of scientific computing packages preinstalled (numpy, scipy, pandas, sci-kit learn, etc.) so you have access to all of them without having to install them all one by one yourself. If you have a problem you want solved but there isn't a preinstalled package to do it, you can still look into the Anaconda library, the Conda-forge library or just PyPi to find it.

 

[atyy]

If you are learning python, after downloading the anaconda distribution as Matterwave suggests, you can try out http://briansimulator.org/. It's not the fastest, but it is a good place to start learning.

If you learn C++, you can try http://www.nest-simulator.org/ (I've never used it, but many people do, especially those who need speed).

For mathematics, you need differential equations, and probability (and statistics). Linear algebra is also useful, as it is applicable to both differential equations (eg. solving a linear differential equation), and to probability (eg. principal components analysis).

 

[jedishrfu]

There’s also the site

https://lectures.quantecon.org/

Which has lectures on python and Julia.

 

[Eagle9]

Matterwave
atyy
jedishrfu
Thanks a lot

I suggest getting the anaconda distribution

after downloading the anaconda distribution

I do not completely understand - anaconda distribution is the part of Python language?

 

[Matterwave]

Matterwave
atyy
jedishrfu
Thanks a lot I do not completely understand - anaconda distribution is the part of Python language?

It's a distribution of Python...Python is the actual computing language. Anaconda will come with a command prompt to run Python scripts as well as a bunch of different python packages pre-installed. See here: https://www.anaconda.com/download/

 

[jedishrfu]

You can use the conda command to search or to get additional python packages. It takes up a fair amount of space on your disk. It’s great for development but not necessarily for production deployment.