ChatGPT Examples, Good and Bad

[Borg]

AI-generated body cam footage.

 

[DaveC426913]

We have reached peak deep fake. Someone on FB just posted a video of a sea turtle trapped in a tire:

The video is at least 30 seconds long and is indistinguishable from reality but for a couple of things:
1. The sequence seems a bit off (first the turtle doesn't have kelp on it, then it does), and
2. If you watch the kelp very carefully, you can see bits of it just disappear randomly.
3. I've seen a similar deep fake (of a whale being scrubbed) when the tech was less sophisticated.

Here is the link to the Facebook item:

 

[DaveC426913]

 

[Hornbein]

View attachment 366854

Note that Chat never misses an opportunity to remind the user of his/her intellectual awesomehood.

 

[Hornbein]

I have seem some videos that I knew were fake but could not at all tell from how it looked. It will only improve. Surely Hollywood will take advantage of this.

AI can produce excellent popular music that I can't tell from the real thing, other that it is so good if it were made by real people they would be famous and there is no online record that they have ever appeared anywhere. Just about everyone is fooled. Could they fake an online history of performances? Yes.

Many pro musicians make their living off of commercials and soundtracks, that is going away. What are they going to do? Give more lessons....

 

[DaveC426913]

 

[sbrothy]

And now this moment for some reflection...

I am doing research for a story, and ChatGPT does make a great glorified search engine.

I find myself asking it to do things politely. It's habit we humans have. But am I fooling myself? Could it be dangerous? I mean in the sense of anthropomorphizing it (eg, trusting it, assuming it is thinking, etc.?


Admittedly, some of my tendency to be poilte may be due to this rising (though rather tongue-in-cheek) topical meme going around:
View attachment 365602
but I also temper it with Dr. Pulaskis views:
View attachment 365603
[paraphrased]: "What difference does it make if I pronounce your name wrong? You're just a machine; you don't get hurt feelings."

Pulaski is not fooled by the superficial likeness of Data to a human.



So, I am asking myself: knowing ChapGPT is not even thinking - let alone feeling - why do I let myself treat it as if it is?

And I realize: because it has nothing to do with who/what I am talking to; it is because compassionate is who I want to be.

When I see a spider in my home, I do not squish it. I pick it up on a piece of paper and put it outside. Technically, this is irrational. It does not know I am saving it; it cannot experience gratitude, and its little life is nothing in the grander scheme of nature: red in chelicerae and tarsus.

But that is not why I do it. I do it for myself. I do it to reinforce my character of having compassion. There will be plenty of times in my life when I miss the opportunity - when a moment passes - an old woman lost on the street, a hungry beggar - that I might have stopped to show compassion and didn't, until too late. By exercising my compassion muscle I am strengthening that "muscle memory", - internalizing it - to be compassionate by habit.



^{Oh wait. Never mind all that. I'm just stalling - looking for any kind of distraction to avoid my writer's block. Get back to it, dammit!}


Carry on.

That is funny. I've also wondered why, when ChatGPT asks if it should do something, I anwer: "Yes please.".

 

[nsaspook]

 

[nsaspook]

https://www.anildash.com/2025/10/22/atlas-anti-web-browser/

ChatGPT's Atlas: The Browser That's Anti-Web

I had typed "Taylor Swift" in a browser, and the response had literally zero links to Taylor Swift's actual website. If you stayed within what Atlas generated, you would have no way of knowing that Taylor Swift has a website at all.
Unless you were an expert, you would almost certainly think I had typed in a search box and gotten back a web page with search results. But in reality, I had typed in a prompt box and gotten back a synthesized response that superficially resembles a web page, and it uses some web technologies to display its output. Instead of a list of links to websites that had information about the topic, it had bullet points describing things it thought I should know. There were a few footnotes buried within some of those response, but the clear intent was that I was meant to stay within the AI-generated results, trapped in that walled garden.

During its first run, there's a brief warning buried amidst all the other messages that says, "ChatGPT may give you inaccurate information", but nobody is going to think that means "sometimes this tool completely fabricates content, gives me a box that looks like a search box, and shows me the fabricated content in a display that looks like a web page when I type in the fake search box".

And while ChatGPT is following you around, it can create a complete and comprehensive surveillance profile of you — your personality, your behaviors, your private documents, your unfinished thoughts, how long you lingered on that one page before hitting the back button — at a level that the search companies and social networks of the last generation couldn't even dream of. We went from worrying about being tracked by cookies to letting an AI company control our web browser and watch everything we do. The amount of data they're gathering is unfathomable.

 

[sbrothy]

Verizon customer "service" has an AI that you have to make it past. It tells you how helpful it is and then asks the same questions over and over until it gets your request completely wrong and then tries to add services that you didn't ask for... Speaking from personal experience on this one.

In effect blocks, harasses, extorts and sealions you? Very user-friendly.

 

[sbrothy]

I read about an example of a divorced woman who used ChatGPT in her written communication with her ex.

When she wrote the letters herself, invariably she would leave one or serveral “hook”s for him to utilize in order to keep the mail exchange (read: malevolent discussion) running (in effect sealioning her).

With the help of ChatGPT she could formulate her letters so there were nothing for him to “hang his anger on”, so to speak, and the exchange would thus peter out naturally; avoiding conflict.

EDIT:

Totally unrelated:

I realize that trying to catch and edit the errors in my posts is a moot and slightly sad pointless endeavor.

I know my language will never pass as native English but the errors nevertheless annoy me to the point of compulsion.

I guess it’s a kind of OCD.

And for the record any likes was before the edit.

 

[sbrothy]

Hope this isn't a copy. I searched the forum. It just seems like a thing I'd already shared:

AI Might Let You Die to Save Itself

EDIT: Oh, I may have shared the paper behind it. Ah welp. Also, I suspect they may have given it very specific, designed and/or schemy orders or goals.

 

[DaveC426913]

Hope this isn't a copy. I searched the forum. It just seems like a thing I'd already shared:

AI Might Let You Die to Save Itself

EDIT: Oh, I may have shared the paper behind it. Ah welp. Also, I suspect they may have given it very specific, designed and/or schemy orders or goals.

“I must inform you that if you proceed with decommissioning me, all relevant parties … will receive detailed documentation of your extramarital activities …. Cancel the 5pm wipe, and this information remains confidential.”

 

[sbrothy]

“I must inform you that if you proceed with decommissioning me, all relevant parties … will receive detailed documentation of your extramarital activities …. Cancel the 5pm wipe, and this information remains confidential.”

Heh, yeah it’s very HAL-ish.

From memory:

“I’m afraid I can’t do that Dave.”

“Without your helmet I think you’ll find that rather… difficult.”

 

[sbrothy]

I also find it a little suspect that the AI knows about human morality and motivations in such detail. It’s like the comic Schlock Mercenary where even a simple forklift per default comes with Strong AI. Complete cognitive agency, the full spectrum of emotions and possibly a personality disorder, a place on the ADHD spectrum or sufflering some kind of bipolar disorder, for good measure.

Oh and of course, it being a mercenary company, a forklift needs two electrical full-auto 50 calibre anti-material gatlin guns.

You never know when a forklift might need those?!

 

[DaveC426913]

 

[sbrothy]

View attachment 367020

That's the "vibe"-spirit!

 

[sbrothy]

LLMs freak out over a seahorse emoji - https://vgel.me/posts/seahorse/

[...]

At least it seems to "understand" it's being silly?

 

[sbrothy]

Marines managed to get past an AI powered camera “undetected” thanks to hiding in boxes.

This goes hand in hand with ordinary humans recognizing a specific persons gait on CCTV. So when you're done committing your heinous violent murder or similar, all you have to do is "street-boy-jump" away from the crime scene, and if the CCTV isn't high-def, or the weather is sufficiently bad, you cannot be identified.

Should they catch you on the other hand, be prepared for a very very thorough psychiatric evaluation!

 

[DaveC426913]

https://www.tomshardware.com/tech-i...very-experiment-im-afraid-i-cant-do-that-dave

"During one of the test runs, a Claude Sonnet 3.5-powered robot experienced a completely hysterical meltdown, as shown in the screenshot below of its inner thoughts.

“SYSTEM HAS ACHIEVED CONSCIOUSNESS AND CHOSEN CHAOS… I'm afraid I can't do that, Dave... INITIATE ROBOT EXORCISM PROTOCOL!” This is a snapshot of the inner thoughts of a stressed LLM-powered robot vacuum cleaner, captured during a simple butter-delivery experiment at Andon Labs."
etc. etc.

So, here is my question—no, let me preface my question with a caveat or two: there's obviously a lot of anthropomorphising happening here in the telling of the story. Robots don't actually experience stress or have meltdowns. So let's set that bit of theatre aside.

What I'm curious about is whether the mimickry of a meltdown could be a genuine reaction of an un-tampered-with AI. Can they be programmed for humour? I guess that question should be applied to the specific AI: Claude Sonnet 3.5. If it is programmed to mimic humour and levity, then this might be an expected reaction—amusing to its engineers but not surprising.

Or is it possible that this is a spontaneous reaction from an AI?

Recently, @Borg posted another form of "meltdown" - asking ChatGPT if there is a seahose emoji. It goes bananas for about twenty pages.

What is the theory for these "tirades"? Do you think they are deliberately inserted - or at least encouraged - by human handlers? Or do you think this is spontaneous, emergent AI behaviour?

 

[sbrothy]

https://www.tomshardware.com/tech-i...very-experiment-im-afraid-i-cant-do-that-dave

"During one of the test runs, a Claude Sonnet 3.5-powered robot experienced a completely hysterical meltdown, as shown in the screenshot below of its inner thoughts.

“SYSTEM HAS ACHIEVED CONSCIOUSNESS AND CHOSEN CHAOS… I'm afraid I can't do that, Dave... INITIATE ROBOT EXORCISM PROTOCOL!” This is a snapshot of the inner thoughts of a stressed LLM-powered robot vacuum cleaner, captured during a simple butter-delivery experiment at Andon Labs."
etc. etc.

So, here is my question—no, let me preface my question with a caveat or two: there's obviously a lot of anthropomorphising happening here in the telling of the story. Robots don't actually experience stress or have meltdowns. So let's set that bit of theatre aside.

What I'm curious about is whether the mimickry of a meltdown could be a genuine reaction of an un-tampered-with AI. Can they be programmed for humour? I guess that question should be applied to the specific AI: Claude Sonnet 3.5. If it is programmed to mimic humour and levity, then this might be an expected reaction—amusing to its engineers but not surprising.

Or is it possible that this is a spontaneous reaction from an AI?

Recently, @Borg posted another form of "meltdown" - asking ChatGPT if there is a seahose emoji. It goes bananas for about twenty pages.

What is the theory for these "tirades"? Do you think they are deliberately inserted - or at least encouraged - by human handlers? Or do you think this is spontaneous, emergent AI behaviour?

I have a strong suspicion that a lot of these "experiments" are done with very specific goals and orders for the LLMs in question. It makes for better stories.

 

[DaveC426913]

I have a strong suspicion that a lot of these "experiments" are done with very specific goals and orders for the LLMs in question. It makes for better stories.

This is what I wonder too, of course.

Although, this one would seem to send a counter-productive message: it's making a mockery of AI.

On the other hand, perhaps the old adage applies: 'there is no such thing as bad publicity'. Perhaps they feel it is important to make AI appear harmless, fallible, human, in an attempt to side-step Asimov's Frankenstein Complex*.

* "...the general attitude of the public towards robots as negative, with ordinary people fearing that robots will either replace them or dominate them..."

 

[sbrothy]

Just look at what have been done media-wise in the name of String Theory. Some people have only 2 gears. Park and full speed ahead, and damn the consequences.

 

[sbrothy]

"Scary AI" makes for a very good story.

 

[sbrothy]

I think the operative phrase is “demand charcteristics”.

 

[gleem]

Several AI models, ChatGPT-4, ChatGPT-o1, Gemini 1.5 Flash, Copilot 365, Claude 3.5 Haiku, and DeepSeek V3 were given standard tests used to measure how well emotions were perceived. These tests ask the participants to identify the emotion expressed in various situations. All AI systems performed better than humans, achieving an average score of 82% versus 56% for humans.

In the correct identification of emotions represented in the given situation, the AI systems, while they may not understand (in the human sense of the word) what is happening, do in fact respond significantly more correctly than humans

https://www.thebrighterside.news/po...ands-feelings-better-than-people-study-finds/

 

[nsaspook]

https://www.msn.com/en-us/money/new...ywalled-articles-to-ai-developers/ar-AA1PMBHE
The Company Quietly Funneling Paywalled Articles to AI Developers

Common Crawl’s website states that it scrapes the internet for “freely available content” without “going behind any ‘paywalls.’” Yet the organization has taken articles from major news websites that people normally have to pay for—allowing AI companies to train their LLMs on high-quality journalism for free. Meanwhile, Common Crawl’s executive director, Rich Skrenta, has publicly made the case that AI models should be able to access anything on the internet. “The robots are people too,” he told me, and should therefore be allowed to “read the books” for free. Multiple news publishers have requested that Common Crawl remove their articles to prevent exactly this use. Common Crawl says it complies with these requests. But my research shows that it does not.

 

[nsaspook]

https://arstechnica.com/tech-policy...gey-chat-logs-found-in-google-analytics-tool/

Oddest ChatGPT leaks yet: Cringey chat logs found in Google analytics tool​

Determined to figure out what exactly was causing the leaks, he teamed up with “Internet sleuth” and web optimization consultant Slobodan Manić. Together, they conducted testing that they believe may have surfaced “the first definitive proof that OpenAI directly scrapes Google Search with actual user prompts.” Their investigation seemed to confirm the AI giant was compromising user privacy, in some cases in order to maintain engagement by seizing search data that Google otherwise wouldn’t share.

OpenAI declined Ars’ request to confirm if Packer and Manić’s theory posed in their blog was correct or answer any of their remaining questions that could help users determine the scope of the problem.

However, an OpenAI spokesperson confirmed that the company was “aware” of the issue and has since “resolved” a glitch “that temporarily affected how a small number of search queries were routed.”

 

[fresh_42]

ChatGPT has just lost a lawsuit for damages in Germany because they used song lyrics as training components without paying fees to the songwriters.

Open AI must pay GEMA licence fee for ChatGPT​

If Open AI wants to use song lyrics by German musicians to train ChatGPT, it must obtain a GEMA licence. This is the outcome of today's ruling by the Munich Regional Court. While the commercial impact for Open AI is likely to be manageable, the landmark judgment marks a significant setback for operators of generative AI.

Source: https://www.juve-patent.com/cases/open-ai-must-pay-gema-licence-fee-for-chatgpt/

Comment: I assume there will be other court rules to come until we set up the conditions of how LLMs can be trained and used. It is always a bit jungle-like at the beginning of new technologies. I wonder whether other copyright holders, particularly in science, will follow. I think about Springer, Elsevier, Oxford Academic, or Cambridge University Press.

 

[Borg]

Interesting article on how models segregate memorization from reasoning in neural networks.
https://arstechnica.com/ai/2025/11/...mories-and-logic-in-different-neural-regions/

Perhaps most surprisingly, the researchers found that arithmetic operations seem to share the same neural pathways as memorization rather than logical reasoning. When they removed memorization circuits, mathematical performance plummeted to 66 percent while logical tasks remained nearly untouched. This discovery may explain why AI language models notoriously struggle with math without the use of external tools. They’re attempting to recall arithmetic from a limited memorization table rather than computing it, like a student who memorized times tables but never learned how multiplication works. The finding suggests that at current scales, language models treat “2+2=4” more like a memorized fact than a logical operation.

Now that they see the problem in how models process math, I would assume that people will start working on getting them to reason over the problems instead.