ChatGPT Examples, Good and Bad

  • Thread starter Thread starter anorlunda
  • Start date Start date
  • Tags Tags
    chatgpt
Click For Summary
Experiments with ChatGPT reveal a mix of accurate and inaccurate responses, particularly in numerical calculations and logical reasoning. While it can sometimes provide correct answers, such as basic arithmetic, it often struggles with complex problems, suggesting a reliance on word prediction rather than true understanding. Users noted that ChatGPT performs better in textual fields like law compared to science and engineering, where precise calculations are essential. Additionally, it has shown potential in debugging code but can still produce incorrect suggestions. Overall, the discussion highlights the need for ChatGPT to incorporate more logical and mathematical reasoning capabilities in future updates.
  • #361
Borg said:
Now that they see the problem in how models process math, I would assume that people will start working on getting them to reason over the problems instead.
I wonder how they will deal with the setup. The equation ##2+2=4## does not hold everywhere, and I haven't seen an example yet of a setup that is sufficiently described. The automatism to assume integers (reals) is all over the place, but it is a requirement, not a property, and nobody mentions it.
 
Computer science news on Phys.org
  • #362
fresh_42 said:
I wonder how they will deal with the setup. The equation ##2+2=4## does not hold everywhere, and I haven't seen an example yet of a setup that is sufficiently described. The automatism to assume integers (reals) is all over the place, but it is a requirement, not a property, and nobody mentions it.
From the article, they noticed some correlations in the landscape for memorized and reasoned processes. Assuming that the correlations hold up, I would guess that the next step would be to adjust the backpropagation step to penalize the sharper memorization landscapes when dealing with math.
 
  • #363
Here's a good LLM example. I have to create a Programming Guide for the architecture that I designed and built this year. Last week, I threw sections of the code at an LLM and asked it to generate the guide. I tossed the results in a doc and hadn't reviewed it much though. This morning, I was working on a new factory method in the code and thought that would be good to add. When I reviewed last week's output, there was a nice section on generating the new factory that I'm working on!
 
  • #364
Borg said:
Now that they see the problem in how models process math, I would assume that people will start working on getting them to reason over the problems instead.
I have a problem with mixing the terms "reasoning" and "LLM". I'm reassured with this quote from the same article:
It’s worth noting that “reasoning” in AI research covers a spectrum of abilities that don’t necessarily match what we might call reasoning in humans. The logical reasoning that survived memory removal in this latest research includes tasks like evaluating true/false statements and following if-then rules, which are essentially applying learned patterns to new inputs. This also differs from the deeper “mathematical reasoning” required for proofs or novel problem-solving, which current AI models struggle with even when their pattern-matching abilities remain intact.
 
  • #365
Pakistani newspaper mistakenly prints AI response with the article.
1763032951110.webp


 
  • Like
Likes DaveC426913
  • #369
fresh_42 said:
It can at least create a typical conversation:

"I didn't know."
"I told you last week, Monday, 3:15:25 pm."
It won't truly be like a wife until it can remind you of the toilet seat that you left up two years earlier. :wink:
 
  • #370
Maybe that's the new Turing test: does the chatbot male partner forget anniversaries, leave the seat up, pretend to listen while watching football...
 
  • #371
gmax137 said:
Maybe that's the new Turing test: does the chatbot male partner forget anniversaries, leave the seat up, pretend to listen while watching football...
I had a story idea kicking around about the idea that the last barrier between androids becoming human ultimately required them to be able to misremember and even forget stuff.

Data, on Star Trek, for example, cannot understand humanity, or become human, until his memory and thought processes are messy and unreliable.

And of course, once that happens, we're removed the big advantage of androids; they're as imprecise as humans.

A sort of 'you can't have your cake and eat it too' kind of thing. (shades of Twilight Zone's 'The Mighty Casey'.)
 
Last edited:

Similar threads

  • · Replies 212 ·
8
Replies
212
Views
14K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 21 ·
Replies
21
Views
3K
Replies
66
Views
7K
Replies
10
Views
4K
Replies
14
Views
464
Replies
4
Views
2K
  • · Replies 4 ·
Replies
4
Views
3K
  • · Replies 3 ·
Replies
3
Views
2K
Replies
9
Views
1K