Modelling Memory Consolidation using Neural Networks

**gadgets** · Sep27-06, 09:39 AM

Hi guys,

If there's anyone out there who has knowledge in this area, I'm seeking to find out how to model memory consolidation using neural networks.

I was thinking of using a Hopfield network to train another Hopfield network. The first network would represent the hippocampus, and the second network would represent the neocortex. I thought this was appropriate since the hippocampus actually acts as a "teacher" to the neocortex.

I'm wondering if my thinking is correct?

Any advice would be greatly appreciated.

**gadgets** · Oct7-06, 02:59 AM

Another some reading, I think the hippocampus should be implemented as Kohonen network, since it performs off-line (unsupervised) learning. And this network will act as a teacher to a hopfield network (neocortex).

Can any experts advise me on this matter?

**gadgets** · Oct10-06, 11:10 PM

Nobody knows?

**neurocomp2003** · Oct11-06, 03:39 PM

many ways you can try it. the outline you listed is worth a try.There are many theories out there....and just as many on how to code it.

search online for a researcher:Sue becker

**gadgets** · Oct23-06, 03:50 AM

Thanks.

Do you know how should information be passed to the other network?

I was thinking, the first Kohonen network (Hippocampus) can be trained with a training set to get the desired weights. How can I use this network to train the 2nd Kohonen network (Neocortex)?

I read about pseudorehearsals but don't quite understand the concept. Does it mean that I should just use random inputs at the Hippocampus or something like that?

**MeJennifer** · Oct23-06, 03:54 AM

Originally Posted by gadgets

I was thinking, the first Kohonen network (Hippocampus) can be trained with a training set to get the desired weights. How can I use this network to train the 2nd Kohonen network (Neocortex)?

What are you trying to achieve with the second Kohonen network?

**gadgets** · Oct23-06, 04:49 AM

Originally Posted by MeJennifer

What are you trying to achieve with the second Kohonen network?

The 2nd Kohonen network is akin to the Neocortex, where all the long term memory is stored. I'm trying to model the concept of consolidation, whereby the Hippocampus learns and transfers the memory to the Neocortex during REM/NREM sleep.

Let me know if I'm going wrong somewhere.

Thanks.

**neurocomp2003** · Oct23-06, 11:07 PM

if you use a kohonen network...it'll be very sparse IMO.
again try it out if it doesn't work then you will know.
You might be interested to read up on a self-motivated researcher
in the UK named Steve Grand(look up his book on "Growing up with Lucy")

**MeJennifer** · Oct24-06, 12:00 AM

Originally Posted by gadgets

The 2nd Kohonen network is akin to the Neocortex, where all the long term memory is stored. I'm trying to model the concept of consolidation, whereby the Hippocampus learns and transfers the memory to the Neocortex during REM/NREM sleep.

Let me know if I'm going wrong somewhere.

Thanks.

Well perhaps I miss something.
Once the first Kohonen network "clusters" the significant statistical coincidences of the input neurons, what could the second one possibly improve on that?

**gadgets** · Oct24-06, 07:35 AM

Originally Posted by MeJennifer

Well perhaps I miss something.
Once the first Kohonen network "clusters" the significant statistical coincidences of the input neurons, what could the second one possibly improve on that?

Actually, I'm still feeling in the dark, trying to figure which is the right way to have it implemented.

I read a few papers and came to know about catastrophic interference, which is the reason why some researchers have proposed dual-network memory models to resolve this problem.

Then I came to wonder about how the first network (hippocampus) could possibly transfer or "teach" the second network (neocortex). I read that Robins proposed the idea of pseudo-patterns. From what I understand, this means creating random inputs to feed to the artificial hippocampus. These pseudo patterns could then be used to train neocortex.

Thus, I had thought of using these two Kohonen networks (maybe a wrong idea), whereby the first one learns and then transfers to the 2nd.

If there're any experts in this area around, I'd appreciate any comments.

**gadgets** · Oct24-06, 07:42 AM

Perhaps a more correct way would be to use a Hopfield Network as the hippocampus to perform the initial learning. The neocortex could be implemented as an MLP. Thus, during a consolidation phase, random inputs could be fed to the Hopfield to obtain the trained outputs. These values could then facilitate the training of the MLP (neocortex).

I'm still trying to figure out how catastrophic interference is minimized in this scenario.

Pardon my poor knowledge/understanding in this area. Trying my best to make sense of the whole thing. Getting a little upset - must be my stupid amygdala at work.

**gadgets** · Oct24-06, 08:04 AM

Originally Posted by neurocomp2003

if you use a kohonen network...it'll be very sparse IMO.
again try it out if it doesn't work then you will know.
You might be interested to read up on a self-motivated researcher
in the UK named Steve Grand(look up his book on "Growing up with Lucy")

Interesting biography. It's motivating to see an individual with so much drive and self-initiative to delve into such a complex area of study.. It's a pity no one is funding the Lucy project anymore.