ChatGPT spin off from: What happens to the energy in destructive interference?

[ergospherical]

The human brain doesn’t have an explicit “fact model”, but neural connections are formed during life-experiences that are effectively its training data. It’s quite interesting to observe some parallels between LLMs and the brain — both are subject to biases, learning falsehoods, etc.

The human brain clearly isn’t useless…

It’s fair enough to be skeptical of issues with the models (there are many!), but to try and downplay just how impressive some of the most cutting edge LLMs are is pretty disingenuous.

 

[Dale]

The human brain doesn’t have an explicit “fact model”, but neural connections are formed during life-experiences that are effectively its training data.

It absolutely does. That is the whole purpose of the neurosensory system. Yes, training is critical for the neurosensory system and that system is also used to train much more flexible and abstract networks of neurons. But the neural connection to our senses, our biological fact-gathering organs, is explicit and structural.

The human brain clearly isn’t useless…

Obviously not. I am not sure why you would imply that it is.

 

[Hornbein]

I say that anything that passes tests of intelligence is intelligent.

It's useless to argue over the definitions of words. No one controls this. I just state my opinion then move on.

 

[ergospherical]

It absolutely does. That is the whole purpose of the neurosensory system. Yes, training is critical for the neurosensory system and that system is also used to train much more flexible and abstract networks of neurons. But the neural connection to our senses, our biological fact-gathering organs, is explicit and structural.

In what sense is this not comparable to the transformer architecture in an LLM (the huge association cortex trained via reinforcement learning on enormous training sets)?

 

[Astronuc]

It's useless to argue over the definitions of words.

A semanticist might disagree, although perhaps it might be more a debate than an argument. Semanticists study the meanings in language, i.e., the meanings (and definitions) of words, phrases and the context in which they are used. In theory, large language models can be 'taught' to do that. However, if examples of poor grammar are used, then the AI software may produce erroneous results.

No one controls this.

While true, we need to have an agreement or consensus as to the definition of words and their meaning in a give context, otherwise accurate communication is compromised and misunderstandings are likely.

 

[Dale]

In what sense is this not comparable to the transformer architecture in an LLM (the huge association cortex trained via reinforcement learning on enormous training sets)?

In the following senses:

1) there is no factual input to the LLM. I.e. nothing similar to eyes or skin, even abstractly.
2) there is no part of the LLM that organizes the factual input into a world model. I.e. no visual cortex and no somatosensory area.
3) there is no part of the LLM that connects the language model to the world model

I assume that at some point developers will connect a LLM like GhatGPT to a neuro symbolic AI like Watson. The resulting software will be more than a LLM.

 

[Mondayman]

In a general knowledge contest between you and ChatGPT you would not have a hope! It would be completely unbeatable even by the best human quiz expert.

Not entirely true. I used to use ChatGPT for hockey/American football trivia and had to stop because it simply sucked and couldn't compete with my encyclopedic knowledge of useless sports facts.

 

[PeroK]

Not entirely true. I used to use ChatGPT for hockey/American football trivia and had to stop because it simply sucked and couldn't compete with my encyclopedic knowledge of useless sports facts.

That is not general knowledge. That is a specialist subject.

 

[russ_watters]

It's useless to argue over the definitions of words.

I agree, but what is important is understanding what its strengths and limitations are, if you're going to use it or make an informed choice not to.

 

[pbuk]

I am amazed we are having this discussion again. @PeroK nobody is disputing that Chat GPT often produces answers that are correct. However it also produces answers that are not correct, furthermore it [Chat GPT < 4o1] has no way of determining internally whether an answer it has produced is correct or not: its design does not include any concept of logical deduction and so it cannot test the steps it has used to arrive at an answer which, by any reasonable definition, means that its answers are unreliable. @PeroK your consistent denial of these facts does not do you credit.

But there is a big BUT here. Chat GPT 4o1, aka Strawberry, does include algorithms that are intended to review the linguistic connections it has made in producing its output, and refining or discarding those connections if it decides they may be incorrect. ChatGPT 4o1 has been on limited release since September 2024 in the form of 'o1-mini' and 'o1-preview'.

Even so, Chat GPT o1 does not detect logical flaws 100% of the time. And even when it does detect that its own output may be unreliable it may simply ignore that and output it anyway without any indication of its uncertainty: see for example this paper and this article on The Verge (note: neither of these sources pass PFs normal test for references: they are the best I can do for the moment).

In conclusion:

• Q. Can systems based on large linguistic models (LLMs) be useful?
  A. Absolutely.
• Q. Are LLMs that are available in October 2024 capable of producing output that is nonsense without any indication that it may be unreliable?
  A. Yes, but they are getting better.

 

[PeroK]

nobody is disputing that Chat GPT often produces answers that are correct.

Um ... that's precisely what they are doing. That's the only thing I've been arguing. That it doesn't "almost always give the wrong factual answer". Which was the original claim.

It's nothing to do with AI or thinking. It's a dispute about whether ChatGPT is almost always wrong.

 

[Dale]

Um ... that's precisely what they are doing. That's the only thing I've been arguing. That it doesn't "almost always give the wrong factual answer". Which was the original claim.

My specific claim is that it is unreliable for factual information. I don’t claim “almost always” wrong, I claim “often” wrong.

 

[Borg]

The OP did claim that though.

Every time I ask ChatGPT something factual, I ask it something that I can check myself, and the answer is almost always factually incorrect.

 

[Dale]

The OP did claim that though.

Yes. They did.

 

[Swamp Thing]

Edit: actually the answer is still wrong, but not as wrong as before.

At least it isn't "not even wrong".

 

[Tom.G]

This thread is sounding an awful lot like arguing about "How many fairies can dance on the head of a pin."