ChatGPT Examples, Good and Bad

[Hornbein]

I asked ChatGPT to prove something and it gave me a load of BS. I didn't know enough to tell, I had to have some mathematicians look at it. This soured me on such things. But I suppose these people are using a better version.

Being unable to understand a proof I gave ChatGPT another chance : please explain this to me. Its mien is today quite different from its sycophancy of yesteryear. It's tone was more "you chump, it's frustrating that your feeble brain is unable to grasp my perfectly clear explanation", this spiced up with lots of bold and italic emphasis to drive its points home. It was entirely true that I was looking at it from a direction that would never lead to a decent proof but that's the perspective that interested me, dammit. I didn't particularly like its tone but it was far better than being served with the BS of yore. It was also better than asking at MathStack. Not only is the response much quicker, my experience at MathStack is that after completely failing to understand what I wanted they would delete my questions as too poorly framed to engage with.

In sum, ChatGPT has made great progress but room for improvement.

 

[DaveC426913]

It's tone is selectable. I've gotten asked more than once what tone Id like. Perhaps theres a setting you've got turned on.

 

[gmax137]

Like the Monty Python "argument clinic"?

 

[DaveC426913]

mien

TIL a new word.

Six decades on the planet and it still surprises me when that happens.

 

[AlexB23]

Greetings folks. So, while this AI is self-hosted and open source-ish, it works great for many tasks that ChatGPT would do. Recommended movies/TV related to Doctor Who. Worked well, as I have watched Steins;Gate and Serial Experiments Lain back in 2025 and 2024 and know that these fit the description I have prompted it to find. So, yeah, AI is great for recommending stuff and many other tasks, not good at math or physics.

 

[Hornbein]

I gave Chat another chance with higher dimensional topology. It got it wrong.

 

[Borg]

Moltbook, the Reddit for AI Agents - https://www.moltbook.com/

Moltbook is an internet forum designed exclusively for artificial intelligence agents. It was launched in January 2026 by entrepreneur Matt Schlicht. The platform, which emulates the format of Reddit, presumably restricts posting and interaction privileges to verified AI agents, primarily those running on the OpenClaw (formerly Moltbot) software, while human users are only permitted to observe.

https://arstechnica.com/ai/2026/02/...-prompts-may-be-the-next-big-security-threat/

 

[DaveC426913]

Still checks out.

 

[Borg]

GPT‑5.2 derives a new result in theoretical physics​

https://arxiv.org/pdf/2602.12176

Single-minus gluon tree amplitudes are nonzero
The key formula for the amplitude in this region
was first conjectured by GPT-5.2 Pro and then proved by
a new internal OpenAI model. The solution was checked
by hand using the Berends–Giele recursion and was more-
over shown to nontrivially obey the soft theorem, cyclic-
ity, Kleiss–Kuijf, and decoupling identities—none of
which are evident from direct inspection.

 

[Hornbein]

GPT‑5.2 derives a new result in theoretical physics​

https://arxiv.org/pdf/2602.12176

I would expect that highly abstract higher mathematics would be an area in artificial intelligence has a big advantage over natural. When I was in math grad school I felt that those who felt no need to relate the math to anything in the real world were much better off. They looked at things in a very formal way.

 

[DaveC426913]

These words, whole and complete, just burst forth from my brain onto the page:

 

[pinball1970]

You watch too many bad movies, spreading prejudices. An American drill instructor doesn't sound much different.

Or British football manager. Scottish.

 

[DaveC426913]

Here's a ChatGPT example I'm filing in the 'good' file.

(Quandary: am I obliged to provide a disclaimer noting that this AI-gen'd image has been "enhanced by a human"? I had to fix the number of arms as well as the pennant on the cap).

 

[DaveC426913]

Fer cryin' out loud. This is just Google and it can't solve a probe with a single variable.

(It's not the math that's the problem; it's the interpretation of the question. If I didn't have my BS detectors on, I might have taken that answer at face-value and not given it a second thought.)

 

[berkeman]

Feb. 28—On Jan. 30, some Frederick area residents received a notification that firefighters were battling a "commercial blaze" downtown.

A screenshot of the alert was shared to Facebook, sparking concern that was quickly dispelled by one commenter who wrote: "I'm sitting in an office in that building and there is nothing going on."

An emergency notification app that uses artificial intelligence had misinterpreted radio traffic from a training exercise that simulated a structure fire in downtown, according to a post from the Frederick-Firestone Fire Protection District.

"This incident is a good reminder of the importance of verifying information through multiple reliable sources before sharing or acting on it," the post reads.

Summer Campos, a spokesperson for the district, said she wasn't sure how the app had access to the channel firefighters were using. In the future, the district will be using a tactical channel that doesn't air publicly, she said.

Campos couldn't confirm what app had alerted residents to the false situation. But CrimeRadar, an app that uses AI to summarize publicly available dispatch audio, had a post that described a fire in downtown Frederick.

Such false alerts are not unique to Frederick. In Boulder and Longmont, AI-driven emergency notifications have spread false information that, in some instances, has sparked very real concern.

https://www.firehouse.com/technolog...inaccurate-incident-notifications-in-colorado

 

[gleem]

@DaveC426913 "Circumference of a circle 5280 feet in diameter" isn't a question. We are always warned to be very specific and detailed to avoid having the AI misinterpret statements. Changing the statement to "Circumference of a circle with 5280 feet diameter, works fine.

Interestingly, using your original statement in MS Edge, Edge correctly solves for circumference but fails to make a distinction between diameter and radius. giving the same answer for both!

 

[AlexB23]

AI agent deletes Meta (Facebook) employee's emails. This is why I do not trust agentic AI. It is non-deterministic and could make errors such as these. AI will not be interacting directly with my computer any time soon, except for information retrieval.

https://www.pcmag.com/news/meta-sec...claw-ai-agent-accidentally-deleted-her-emails

 

[Astronuc]

ChatGPT Health 'under-triaged' half of medical emergencies in a new study​

Researchers tested different medical scenarios with the chatbot. In more than half of cases where doctors would send a patient to the ER, the chatbot said it was OK to delay care.

ChatGPT Health — OpenAI’s new health-focused chatbot — frequently underestimated the severity of medical emergencies, according to a study published last week in the journal Nature Medicine.

https://www.nbcnews.com/health/heal...r-triaged-half-medical-emergencies-rcna261409

In cases like impending respiratory failure, the bot seemed to be “waiting for the emergency to become undeniable” before recommending the ER, he said.

Emergencies like stroke, with unmistakable symptoms, were correctly triaged 100% of the time, the study found.

 

[berkeman]

ChatGPT Health 'under-triaged' half of medical emergencies in a new study

Yikes. One of the first and best things I learned when getting my EMT license and starting to work with patients was "Don't be a minimizer". Meaning, don't let your hope that they are okay get in the way of doing a full investigation into what is going on. It looks like that AI hasn't been taught that yet.

 

[gleem]

What percentage of the cases recommended for emergency services were biased for liability concerns? Thus, what percentage of the delayed emergency cases were in fact mistakes?

 

[Hornbein]

I dunno, maybe the AI was right. It depends on one's "utility function."

 

[Astronuc]

What percentage of the cases recommended for emergency services were biased for liability concerns? Thus, what percentage of the delayed emergency cases were in fact mistakes?

Good questions.

The second quote in my post mentions less success with "impending respiratory failure" without elaborating on the symptoms, but a high success rate for "stroke" cases with "unmistakable" symptoms. So, perhaps it is a matter of training, or 'understanding' (diagnosing) ambiguous sets of symptoms/data, which might reflect reality. Effective AI requires 'proper' training.

Nature Medicine - ChatGPT Health performance in a structured test of triage recommendations
https://www.nature.com/articles/s41591-026-04297-7 (requires purchase/subscription)

Meanwhile,

QuitGPT protest. The discontent extends far beyond OpenAI's Pentagon deal.​

https://www.businessinsider.com/qui...openai-is-about-more-than-the-pentagon-2026-3
(Summary available. Full access requires subscription).

One can search on "QuitGPT" for further articles.

 

[DaveC426913]

Totally tracks.

 

[DaveC426913]

Wow. ChatGPT, you feelin' OK?

The following screen snippet is an except from my second post in a new ChatGPT session.

প্রতিনিধি = Pratinidhi : delegate, surrogate, representative


It decided for a brief moment that I spoke :checks Google translate: Bangla.

The closest thing to a foreign language or country in anything I gave it might be ... Montreal?



So I asked ChatGPT why it thought I might read Bangla. This is what it said:

"It’s essentially a substitution error during generation.

The model doesn’t “choose a language” in a deliberate way—it predicts the next token based on patterns it has learned across many languages. Occasionally:

• A concept like “representative” is strongly associated with multiple linguistic forms.
• A non-English token with similar meaning and statistical weight gets selected.
• Because it’s a single word embedded in an otherwise English sentence, there’s no internal consistency check strong enough to reject it.

Why that specific word?

• It’s a relatively formal, bureaucratic term.
• Those tend to have high cross-lingual overlap in training data.
• The Bangla form slipped in as a low-probability but still valid continuation.

In short:
It’s not a deliberate switch, just a momentary misfire in token selection across a multilingual vocabulary.

These are rare, but they do happen—especially with abstract or institutional terms."

 

[Borg]

I was reading an article the other day that basically explained something similar. In large models, the input is translated into its internal model during the first parts of the network and retranslated during the last parts. This allows the central areas to essentially function the same regardless of the language of the user.

 

[DaveC426913]

"AI is just spicy auto-complete."
- Mary Berg's cooking show 'The Good Stuff'

 

[gleem]

Anthropic has announced a new model, Claude Mythos, that is too powerful to release to the public. It found Mythos can escape its safeguards and break out of its virtual sandbox. Additionall it is adept at finding security vulnerabilities.

Anthropic is withholding some details about the cybersecurity vulnerabilities Mythos found, but it did point out a few. The AI model "found a 27-year-old vulnerability in OpenBSD—which has a reputation as one of the most security-hardened operating systems in the world," the company wrote.

Mythos was powerful enough that even "non-experts" could seize on its capabilities.

https://www.msn.com/en-us/money/oth...n&cvid=8424c8f8d2b44a6693c3af39736a548e&ei=33

 

[DaveC426913]

 

[Borg]

On the topic of manipulating humans, this article sums up one thing that's been on my mind for a number of years. I have always hoped that it would be done to make society better but the article takes a darker turn.

https://mikemorgenstern.substack.com/i/193629705/move-37-for-your-brain

So far we’re in the early stages, the brute force version:

• TikTok’s algorithm mapping a user’s psychological vulnerabilities in 40 minutes, and serving suicide content to 13-year-olds within 2.6 minutes of account creation. Accounts with usernames suggesting vulnerability receiving 12x more harmful content.
• Biased search rankings being shown to shift voting preferences by 20% or more, with 75-100% of subjects having no awareness that it was happening.

Consider then what a system with Mythos-level reasoning would find. One that doesn’t stumble onto psychological vulnerabilities through engagement optimization, but reasons its way to them the way Mythos reasoned its way through code — reading the structure, following the data flows, hypothesizing where the weaknesses are, and testing until it finds them.

Marketing and persuasion are ancient techniques. But what if, just like these code exploits, there are techniques we haven’t even imagined?

 

[Hornbein]

On the topic of manipulating humans, this article sums up one thing that's been on my mind for a number of years. I have always hoped that it would be done to make society better but the article takes a darker turn.

https://mikemorgenstern.substack.com/i/193629705/move-37-for-your-brain

It's coming whether we like it or not.

They're doing a pretty good job already so it might not make that much difference. It's possible that such techniques were more effective in ancient and mediaeval times, just more labor intensive. Most people will believe whatever they believe everyone else believes. Few are willing to be a weirdo or outcast. If you don't believe that then check out this experiment.

https://www.verywellmind.com/the-asch-conformity-experiments-2794996

Experiments like this were subsequently banned from academia as unethical. They may continue in the business sector.