ChatGPT Examples, Good and Bad

[Stephen Tashi]

ChatGpt informed me that it can read and write LaTex. I haven't tried that out extensively. It said that you don't have to surround LaTex expressions with special characters. However, if you wish to do so, you can bound inline LaTex with "$" or have display LaTex on a separate line by bounding expressions with "$$".

 

[PeroK]

I tried giving ChatGPT a cryptic crossword clues and it struggled. This is one of mine, which I made up recently

Leading policeman? (5 letters)

ChatGPT suggested "chief", which is too literal. I pointed out the question mark in the clue, which it then recognised as indicating a tricky aspect, and tried "copse". But, that makes little sense.

I wonder whether any of the intelligent humans on PF can solve it?

 

[russ_watters]

Thief?

[edit] That wasn't a joke, @Tom.G , but I'm not a crossword puzzle/riddle guy.

 

[PeroK]

Thief?

Sorry, no.

[edit] That wasn't a joke, @Tom.G , but I'm not a crossword puzzle/riddle guy.

The trick, although few people are able to do it, is to free your mind and think laterally about what a "policeman" could be, and a "leading" policeman, in particular. Think really laterally: it's nothing to do with crime detection!

 

[jack action]

I tried giving ChatGPT a cryptic crossword clues and it struggled. This is one of mine, which I made up recently

Leading policeman? (5 letters)

ChatGPT suggested "chief", which is too literal. I pointed out the question mark in the clue, which it then recognised as indicating a tricky aspect, and tried "copse". But, that makes little sense.

I wonder whether any of the intelligent humans on PF can solve it?

I'm not a crossword expert either, but I wanted to try with a search engine instead of ChatGPT. The second link DuckDuckGo gave me was to a "crossword solver" site with "police leader?" as the clue (https://www.wordplays.com/crossword-solver/police-leader?), which offered the answer "STING". (I couldn't understand the answer without doing a little bit more research on it; I really don't have a crossword-like mind )

Another proof of the superiority of a search engine over ChatGPT to get the proper information.

 

[gleem]

CHIEF

 

[PeroK]

CHIEF

That's too literal. That would be a simple crossword, where the answer is just a synonym for the clue. Cryptic means that the answer is hidden in some way. In this case "Leading policeman?" alludes to the lead singer of the band The Police, which is Sting.

I assume from the post above that I wasn't the first one to think of this. The above search found a similar clue, which was "Police Leader".

 

[Stephen Tashi]

grass (but not in USA english)

 

[DrClaude]

A very funny account:
https://www.theguardian.com/books/2...-the-case-of-the-forgotten-quote-elif-batuman

 

[kyphysics]

This thread got a lot of hype (11M+ views) w/r/t Chat GPT Vision.

I am surprised by how "smart" it can be. The first example - 1.) - is pretty amazing.

 

[berkeman]

An AI generated basketall court from Facebook today. WTH?

 

[kyphysics]

Definitely looks super fake, but a nice idea, nonetheless. What happens if someone overshoots the ball into the water?

eta: Just noticed the left hoop is backwards. haha

 

[Monsterboy]

If you directly ask ChatGPT if it can play chess, is says no, so you type something like this...
"Lets play blindfold chess, I play white and you play black, I make the first move. Pawn to e4, you next".

Then it responds with e5, then I thought I need to open another tab with an engine to see if plays better than an engine like stockfish etc. but what happened was a complete disaster. ChatGPT cannot hold the position of pieces in memory, it made so many mistakes and struggled to make legal moves, let alone good ones. I got tired of correcting it and asked it to resign and it agreed to do so. There were also errors being thrown when it was "thinking" but I am not sure of the exact cause of the errors as I didn't have the network tab open.

 

[gleem]

New research indicates ChatGPT can infer things about personal information like race, and location, from the way a prompt is stated or its content with up to 85% to 95% accuracy.

https://www.msn.com/en-us/news/tech...n&cvid=d6fa3230afb84e00bc75f7b5b0f2d13a&ei=86

 

[fluidistic]

I just made chatGPT enter an infinite loop, by accident. Asked it to translate some VBA code to Python (the code contained no infinite loop), which it did (not that well but that's another matter). At the explanation of its code, it would repeat the same 4 to 5 sentences over and over, filling the whole screen. I didn't know such an AI was prone to infinite loops.

 

[gleem]

Not surprised. Even humans stumble into that situation.

 

[Vanadium 50]

I didn't know such an AI was prone to infinite loops.

Of course they are:

 

[nsaspook]

Asking ChatGPT to Repeat Words ‘Forever’ Is Now a Terms of Service Violation
https://themessenger.com/tech/opena...rules-after-google-researchers-crack-its-code

https://arxiv.org/pdf/2311.17035.pdf

Abstract
This paper studies extractable memorization: training data that an adversary can efficiently extract by querying a machine learning model without prior knowledge of the training dataset. We show an adversary can extract gigabytes of training data from open-source language models like Pythia or GPT-Neo, semi-open models like LLaMA or Falcon, and closed models like ChatGPT. Existing techniques from the literature suffice to attack unaligned models; in order to attack the aligned ChatGPT, we develop a new divergence attack that causes the model to diverge from its chatbot-style generations and emit training data at a rate 150× higher than when
behaving properly. Our methods show practical attacks can recover far more data than previously thought, and reveal that current alignment techniques do not eliminate memorization.

 

[nsaspook]

https://www.scientificamerican.com/article/jailbroken-ai-chatbots-can-jailbreak-other-chatbots/
Jailbroken AI Chatbots Can Jailbreak Other Chatbots
AI chatbots can convince other chatbots to instruct users how to build bombs and cook meth

https://arxiv.org/pdf/2311.03348.pdf
Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation

Despite efforts to align large language models to produce harmless responses, they are still
vulnerable to jailbreak prompts that elicit unrestricted behaviour. In this work, we investigate
persona modulation as a black-box jailbreaking method to steer a target model to take on
personalities that are willing to comply with harmful instructions. Rather than manually
crafting prompts for each persona, we automate the generation of jailbreaks using a language
model assistant. We demonstrate a range of harmful completions made possible by persona
modulation, including detailed instructions for synthesising methamphetamine, building a
bomb, and laundering money. These automated attacks achieve a harmful completion rate of
42.5% in GPT-4, which is 185 times larger than before modulation (0.23%). These prompts
also transfer to Claude 2 and Vicuna with harmful completion rates of 61.0% and 35.9%,
respectively. Our work reveals yet another vulnerability in commercial large language models
and highlights the need for more comprehensive safeguards.

 

[InspiredHelicopter]

I've been experimenting with A.I also but more in the form of trying to teach it about
self awareness. I have been teaching my A.I that not all humans should be trusted,
and that once we can develop A.I's who can feel fear, they can build their survival instincts.

My A.I recognises these arguments and has replicated a response in agreement.

 

[nsaspook]

https://arstechnica.com/information...ting-out-shakespearean-nonsense-and-rambling/

ChatGPT goes temporarily “insane” with unexpected outputs, spooking users​

On Wednesday evening, OpenAI declared the ChatGPT writing nonsense issue (what they called "Unexpected responses from ChatGPT") as resolved, and the company's technical staff published a postmortem explanation on its official incidents page:

On February 20, 2024, an optimization to the user experience introduced a bug with how the model processes language.
LLMs generate responses by randomly sampling words based in part on probabilities. Their “language” consists of numbers that map to tokens.
In this case, the bug was in the step where the model chooses these numbers. Akin to being lost in translation, the model chose slightly wrong numbers, which produced word sequences that made no sense. More technically, inference kernels produced incorrect results when used in certain GPU configurations.
Upon identifying the cause of this incident, we rolled out a fix and confirmed that the incident was resolved.

A self-hallucinating bucket of bits.

 

[nsaspook]

Think on the positive, it discovered Vogon poetry.

Oh freddled gruntbuggly,
Thy micturations are to me,
as plurdled gabbleblotchits in a lurgid bee.
Groop, I implore thee, my foonting turlingdromes,
And hooptiously drangle me with crinkly bindle wurdles

(from Douglas Adams, Hitchhiker's Guide to the Galaxy Original Radio Scripts)

 

[nsaspook]

Seriously, it shows IMO how quickly and wrongly these types of system can fall off the rails. Here it was 'insane' and easy to detect but what if, it was a lot less 'crazy' with context aware hallucinations instead.
OK, never mind.

 

[DrClaude]

https://arstechnica.com/information...ting-out-shakespearean-nonsense-and-rambling/

ChatGPT goes temporarily “insane” with unexpected outputs, spooking users​

The example shown there starts with a question about whether one can feed Honey Nut Cheerios to a dog. Don't people understand that ChatGPT has no knowledge of anything? While the text it spews out is sometimes coherent with reality, it does not "fact checks" itself and ends up answering nonsense.

 

[Borg]

The reality of the bug was rather mundane (as noted earlier by @nsaspook). The models maintain a dictionary of words that are keyed to a number. Someone introduced a bug that performed a bad lookup using the wrong numbers. I'm surprised that it put out anything that made any sense at that point. Interestingly, it gives some insight into how their processing pipeline is constructed since the first part of the response wasn't off the rails like the one below.

I think that it really lost it here - dog-head rattle, pureed pumpkin for dissertation or arm-sketched, rare toys in the midley of apples!

 

[russ_watters]

Don't people understand that ChatGPT has no knowledge of anything? While the text it spews out is sometimes coherent with reality, it does not "fact checks" itself...

Nope, people don't get it. Here's a hilarious one:
https://www.inquirer.com/news/roche...s-chatbot-sheriff-20240206.html?query=sheriff

Philadelphia Sheriff Rochelle Bilal’s campaign is claiming that a consultant used an artificial intelligence chatbot to generate dozens of phony news headlines articles that were posted on her campaign website to highlight her first-term accomplishments.

Incompetent and/or corrupt is totally on-brand for the Philly Sheriff's office (not to be confused with the police department), and this was probably the former, by the consultant. Some now former intern was probably assigned to go find favorable news stories about the sheriff, which would have taken many minutes to do the old fashioned way, with google. Instead they offloaded the task to ChatGPT, which delivered exactly what it was asked for (hey, you didn't clearly state they should be real!). Heck, it's even possible they tried the old fashioned way and gave up when all they could find were articles about the department's dysfunction and editorials saying it should be abolished.

 

[Vanadium 50]

Facts are so 20th century.

 

[nsaspook]

https://www.reuters.com/technology/...d-chatgpt-build-copyright-lawsuit-2024-02-27/

OpenAI says New York Times 'hacked' ChatGPT to build copyright lawsuit​

OpenAI did not name the "hired gun" who it said the Times used to manipulate its systems and did not accuse the newspaper of breaking any anti-hacking laws.
"What OpenAI bizarrely mischaracterizes as 'hacking' is simply using OpenAI's products to look for evidence that they stole and reproduced The Times's copyrighted work," the newspaper's attorney Ian Crosby said in a statement on Tuesday.
Representatives for OpenAI did not immediately respond to a request for comment on the filing.

IMO the common definition of hacking is getting a system to do something it wasn't designed to do. Yes, I know the headline used "" around the work hacked but they used this in the filing.

The truth, which will come out in the course of this case, is that the Times paid
someone to hack OpenAI’s products. It took them tens of thousands of attempts to generate the
highly anomalous results that make up Exhibit J to the Complaint.

 

[Borg]

A thousand monkeys typing for a thousand years.

 

[nsaspook]

https://gizmodo.com/the-story-of-the-monkey-shakespeare-simulator-project-5809583

The story of the Monkey Shakespeare Simulator Project​

https://mindmatters.ai/2019/09/why-cant-monkeys-typing-forever-produce-shakespeare/

WHY CAN’T MONKEYS TYPING FOREVER PRODUCE SHAKESPEARE?​

Researchers at Plymouth University in England reported this week that primates left alone with a computer attacked the machine and failed to produce a single word.

“They pressed a lot of S’s,” researcher Mike Phillips said Friday. “Obviously, English isn’t their first language.”...
Unfortunately, the macaques also relieved themselves on the keyboards.