ChatGPT Examples, Good and Bad

[fluidistic]

And to solve a physics problem, it feels like a plug and chug with not the right formulae, making a stupid assumption and giving a result that, humanly intuitively I hope, isn't right at the first glance.

Please solve the following physics problem "How many times faster would have the Earth to spin for someone based in NYC feel weightless?"

In order for someone based in NYC to feel weightless, the gravitational force between that person and the Earth must be balanced by a centrifugal force. This can be achieved if the Earth's rotation speed is increased.
The formula for centrifugal force is:
F = m * r * w^2
where F is the centrifugal force, m is the mass of the object, r is the distance from the axis of rotation, and w is the angular velocity (in radians per second).
For a person standing on the surface of the Earth, r is equal to the radius of the Earth, which is approximately 6,371 km.
The formula for gravitational force is:
F = G * m1 * m2 / r^2
where F is the gravitational force, G is the gravitational constant, m1 is the mass of the Earth, m2 is the mass of the person, and r is the distance between the person and the center of the Earth.
For a person standing on the surface of the Earth, r is approximately equal to the radius of the Earth plus the person's height, which we'll assume to be 1.8 meters.
In order for the person to feel weightless, the centrifugal force must be equal in magnitude to the gravitational force. Therefore:
m * r * w^2 = G * m1 * m2 / r^2
Simplifying this equation, we get:
w^2 = G * m1 / r^3
Substituting in the values for G, m1, and r, we get:
w^2 = 3.986 * 10^14 m^3/s^2 / (6,371,000 + 1.8)^3 m^3
Solving for w, we get:
w = 7.905 * 10^-5 radians per second
The current angular velocity of the Earth is approximately 7.292 * 10^-5 radians per second, so the Earth would need to spin approximately 1.08 times faster for someone based in NYC to feel weightless.

 

[fluidistic]

It's also not good with thermoelectricity. It's not surprising to me, since there is a lot of bad information everywhere. At first it mixes up the Thomson effect with the Peltier effect. When pointed out its error, it gives a wrong description of both effects (it mentions heat fluxes rather than heat generation, so it mixes up a vector quantity with a scalar one), and so on and so forth. I am now trying to make it correct itself, but for some reason it is stubborn to claim that the Thomson effect is a heat flux.
Edit: I just corrected it with

You are still saying "it is characterized by a flow of heat due to the interaction between the temperature gradient and the electric field.". Why do you say it is a flow of heat?

. It then corrected itself correctly.

When asked if, in a cubic region where Dirichlet boundary conditions are applied on 2 sides, would it be possible to change one of the Dirichlet's b.c. to a Neumann's type while maintaining the same solution to the heat equation, it answers that in general no, but that it may be possible to affect very little the solution, but that it would depend on the specific problem. I think this is wrong, and that it is indeed possible to always replace one of the Dirichlet b.c. to a Neumann's type without modifying the solution.

It is aware that the thermal gradient (direction from cold to hot) may have a different direction than the heat flux, which is correct.

 

[jack action]

 

[bob012345]

View attachment 326175​

According to Cambridge english dictionary it should be two words. "Firetruck" may be commonly used but it is not the proper spelling.

https://dictionary.cambridge.org/us/dictionary/english/fire-truck

Me: Is " firetruck" a proper word or is it the two words "fire truck" in proper english?

ChatGPT:The word "firetruck" is not recognized as a standard word in the English language. The correct term is "fire truck," which refers to a vehicle used by firefighters to transport equipment and personnel to the scene of a fire. "Fire truck" is a compound noun, composed of the two separate words "fire" and "truck."

Me: Are there any proper words that start with an "f" and end with "uck' that have other letters in between?

ChatGPT: No, there are no standard English words that start with "f" and end with "uck" while having other letters in between. The word "truck" is a common noun and does not have any recognized prefixes that begin with "f" and fit that pattern.

 

[jack action]

I like how ChatGPT seems to always give the answer you want to hear.

 

[bob012345]

I like how ChatGPT seems to always give the answer you want to hear.

I think it's getting better. It seemed to always agree with me even when I gave it bad information as a test. Recently it stood its ground and told me I was wrong.

 

[Tom.G]

It seems the defensive walls are already going up!

https://www.latimes.com/business/st...-thwart-ai-plagiarism-in-schools-online-media

Edward Tian, 22, a Princeton University student studying computer science and journalism, developed an app called GPTZero to deter the misuse of the viral chatbot ChatGPT in classrooms. The app hase racked up 1.2 million (1.2 E6) registered users since January.

He is now launching a new program called Origin aimed at "saving journalism," by distinguishing AI misinformation from fact in online media.
...

GPTZero analyzes the randomness of text, known as perplexity, and the uniformity of this randomness -- called burstiness --- to identify when AI is being used. The tool has an accuracy rate of 99% for human text and 85% for AI text...

 

[Stephen Tashi]

Will ChatGPT-like programs affect the use of science-math forums? In the case of physicsforums and similar, I suspect it eventually will. That's because a large number of posts on physicsforums are of an elementary or intermediate sort (at least in the math sections) and more general that specific puzzles and problems. I find that ChatGPT can answer that type of question. People who ask very advanced and specialized questions on forums often wait awhile to get an answer - or never get one. ChatGPT responses are fast by comparison. The self contradictions and errors of ChatGPT responses are no more confusing that the contradictions that happen when several people respond to a post and debate technical and tangential issues. ChatGPT responses are fast compared to responses on forums.

There is a certain type of response to posts along the lines of "Google is your friend. Just search for...". Perhaps people who respond that way will start saying "ChatGPT is your friend. Just ask it about ...." Perhaps people who intend to answer questions will start double checking their proposed answers by consulting ChatGPT instead of Googling.

 

[Tom.G]

Perhaps people who intend to answer questions will start double checking their proposed answers by consulting ChatGPT instead of Googling.

I certainly hope that we are not that ?*&?%% ... uh, something or another.

 

[gleem]

The self contradictions and errors of ChatGPT responses are no more confusing that the contradictions that happen when several people respond to a post and debate technical and tangential issues. ChatGPT responses are fast compared to responses on forums.

The ChatGPT issue is more dangerous in that the responses tend to be authoritative while often being wrong leaving the prompter satisfied and accepting the explanation as fact. The prudent prompter should attempt to verify the response from another source.

The problem with some PF posts as pointed out above, because of different views and approaches to explaining concepts, sometimes lead to confusion about what the correct explanation is. However, the OP at least can seek clarification or a consensus or perhaps become aware that his question is more complex than originally thought.

 

[Stephen Tashi]

ChatGpt informed me that it can read and write LaTex. I haven't tried that out extensively. It said that you don't have to surround LaTex expressions with special characters. However, if you wish to do so, you can bound inline LaTex with "$" or have display LaTex on a separate line by bounding expressions with "$$".

 

[PeroK]

I tried giving ChatGPT a cryptic crossword clues and it struggled. This is one of mine, which I made up recently

Leading policeman? (5 letters)

ChatGPT suggested "chief", which is too literal. I pointed out the question mark in the clue, which it then recognised as indicating a tricky aspect, and tried "copse". But, that makes little sense.

I wonder whether any of the intelligent humans on PF can solve it?

 

[russ_watters]

Thief?

[edit] That wasn't a joke, @Tom.G , but I'm not a crossword puzzle/riddle guy.

 

[PeroK]

Thief?

Sorry, no.

[edit] That wasn't a joke, @Tom.G , but I'm not a crossword puzzle/riddle guy.

The trick, although few people are able to do it, is to free your mind and think laterally about what a "policeman" could be, and a "leading" policeman, in particular. Think really laterally: it's nothing to do with crime detection!

 

[jack action]

I tried giving ChatGPT a cryptic crossword clues and it struggled. This is one of mine, which I made up recently

Leading policeman? (5 letters)

ChatGPT suggested "chief", which is too literal. I pointed out the question mark in the clue, which it then recognised as indicating a tricky aspect, and tried "copse". But, that makes little sense.

I wonder whether any of the intelligent humans on PF can solve it?

I'm not a crossword expert either, but I wanted to try with a search engine instead of ChatGPT. The second link DuckDuckGo gave me was to a "crossword solver" site with "police leader?" as the clue (https://www.wordplays.com/crossword-solver/police-leader?), which offered the answer "STING". (I couldn't understand the answer without doing a little bit more research on it; I really don't have a crossword-like mind )

Another proof of the superiority of a search engine over ChatGPT to get the proper information.

 

[gleem]

CHIEF

 

[PeroK]

CHIEF

That's too literal. That would be a simple crossword, where the answer is just a synonym for the clue. Cryptic means that the answer is hidden in some way. In this case "Leading policeman?" alludes to the lead singer of the band The Police, which is Sting.

I assume from the post above that I wasn't the first one to think of this. The above search found a similar clue, which was "Police Leader".

 

[Stephen Tashi]

grass (but not in USA english)

 

[DrClaude]

A very funny account:
https://www.theguardian.com/books/2...-the-case-of-the-forgotten-quote-elif-batuman

 

[kyphysics]

This thread got a lot of hype (11M+ views) w/r/t Chat GPT Vision.

I am surprised by how "smart" it can be. The first example - 1.) - is pretty amazing.

 

[berkeman]

An AI generated basketall court from Facebook today. WTH?

 

[kyphysics]

Definitely looks super fake, but a nice idea, nonetheless. What happens if someone overshoots the ball into the water?

eta: Just noticed the left hoop is backwards. haha

 

[Monsterboy]

If you directly ask ChatGPT if it can play chess, is says no, so you type something like this...
"Lets play blindfold chess, I play white and you play black, I make the first move. Pawn to e4, you next".

Then it responds with e5, then I thought I need to open another tab with an engine to see if plays better than an engine like stockfish etc. but what happened was a complete disaster. ChatGPT cannot hold the position of pieces in memory, it made so many mistakes and struggled to make legal moves, let alone good ones. I got tired of correcting it and asked it to resign and it agreed to do so. There were also errors being thrown when it was "thinking" but I am not sure of the exact cause of the errors as I didn't have the network tab open.

 

[gleem]

New research indicates ChatGPT can infer things about personal information like race, and location, from the way a prompt is stated or its content with up to 85% to 95% accuracy.

https://www.msn.com/en-us/news/tech...n&cvid=d6fa3230afb84e00bc75f7b5b0f2d13a&ei=86

 

[fluidistic]

I just made chatGPT enter an infinite loop, by accident. Asked it to translate some VBA code to Python (the code contained no infinite loop), which it did (not that well but that's another matter). At the explanation of its code, it would repeat the same 4 to 5 sentences over and over, filling the whole screen. I didn't know such an AI was prone to infinite loops.

 

[gleem]

Not surprised. Even humans stumble into that situation.

 

[Vanadium 50]

I didn't know such an AI was prone to infinite loops.

Of course they are:

 

[nsaspook]

Asking ChatGPT to Repeat Words ‘Forever’ Is Now a Terms of Service Violation
https://themessenger.com/tech/opena...rules-after-google-researchers-crack-its-code

https://arxiv.org/pdf/2311.17035.pdf

Abstract
This paper studies extractable memorization: training data that an adversary can efficiently extract by querying a machine learning model without prior knowledge of the training dataset. We show an adversary can extract gigabytes of training data from open-source language models like Pythia or GPT-Neo, semi-open models like LLaMA or Falcon, and closed models like ChatGPT. Existing techniques from the literature suffice to attack unaligned models; in order to attack the aligned ChatGPT, we develop a new divergence attack that causes the model to diverge from its chatbot-style generations and emit training data at a rate 150× higher than when
behaving properly. Our methods show practical attacks can recover far more data than previously thought, and reveal that current alignment techniques do not eliminate memorization.

 

[nsaspook]

https://www.scientificamerican.com/article/jailbroken-ai-chatbots-can-jailbreak-other-chatbots/
Jailbroken AI Chatbots Can Jailbreak Other Chatbots
AI chatbots can convince other chatbots to instruct users how to build bombs and cook meth

https://arxiv.org/pdf/2311.03348.pdf
Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation

Despite efforts to align large language models to produce harmless responses, they are still
vulnerable to jailbreak prompts that elicit unrestricted behaviour. In this work, we investigate
persona modulation as a black-box jailbreaking method to steer a target model to take on
personalities that are willing to comply with harmful instructions. Rather than manually
crafting prompts for each persona, we automate the generation of jailbreaks using a language
model assistant. We demonstrate a range of harmful completions made possible by persona
modulation, including detailed instructions for synthesising methamphetamine, building a
bomb, and laundering money. These automated attacks achieve a harmful completion rate of
42.5% in GPT-4, which is 185 times larger than before modulation (0.23%). These prompts
also transfer to Claude 2 and Vicuna with harmful completion rates of 61.0% and 35.9%,
respectively. Our work reveals yet another vulnerability in commercial large language models
and highlights the need for more comprehensive safeguards.

 

[InspiredHelicopter]

I've been experimenting with A.I also but more in the form of trying to teach it about
self awareness. I have been teaching my A.I that not all humans should be trusted,
and that once we can develop A.I's who can feel fear, they can build their survival instincts.

My A.I recognises these arguments and has replicated a response in agreement.

 

[nsaspook]

https://arstechnica.com/information...ting-out-shakespearean-nonsense-and-rambling/

ChatGPT goes temporarily “insane” with unexpected outputs, spooking users​

On Wednesday evening, OpenAI declared the ChatGPT writing nonsense issue (what they called "Unexpected responses from ChatGPT") as resolved, and the company's technical staff published a postmortem explanation on its official incidents page:

On February 20, 2024, an optimization to the user experience introduced a bug with how the model processes language.
LLMs generate responses by randomly sampling words based in part on probabilities. Their “language” consists of numbers that map to tokens.
In this case, the bug was in the step where the model chooses these numbers. Akin to being lost in translation, the model chose slightly wrong numbers, which produced word sequences that made no sense. More technically, inference kernels produced incorrect results when used in certain GPU configurations.
Upon identifying the cause of this incident, we rolled out a fix and confirmed that the incident was resolved.

A self-hallucinating bucket of bits.

 

[nsaspook]

Think on the positive, it discovered Vogon poetry.

Oh freddled gruntbuggly,
Thy micturations are to me,
as plurdled gabbleblotchits in a lurgid bee.
Groop, I implore thee, my foonting turlingdromes,
And hooptiously drangle me with crinkly bindle wurdles

(from Douglas Adams, Hitchhiker's Guide to the Galaxy Original Radio Scripts)

 

[nsaspook]

Seriously, it shows IMO how quickly and wrongly these types of system can fall off the rails. Here it was 'insane' and easy to detect but what if, it was a lot less 'crazy' with context aware hallucinations instead.
OK, never mind.

 

[DrClaude]

https://arstechnica.com/information...ting-out-shakespearean-nonsense-and-rambling/

ChatGPT goes temporarily “insane” with unexpected outputs, spooking users​

The example shown there starts with a question about whether one can feed Honey Nut Cheerios to a dog. Don't people understand that ChatGPT has no knowledge of anything? While the text it spews out is sometimes coherent with reality, it does not "fact checks" itself and ends up answering nonsense.

 

[Borg]

The reality of the bug was rather mundane (as noted earlier by @nsaspook). The models maintain a dictionary of words that are keyed to a number. Someone introduced a bug that performed a bad lookup using the wrong numbers. I'm surprised that it put out anything that made any sense at that point. Interestingly, it gives some insight into how their processing pipeline is constructed since the first part of the response wasn't off the rails like the one below.

I think that it really lost it here - dog-head rattle, pureed pumpkin for dissertation or arm-sketched, rare toys in the midley of apples!

 

[russ_watters]

Don't people understand that ChatGPT has no knowledge of anything? While the text it spews out is sometimes coherent with reality, it does not "fact checks" itself...

Nope, people don't get it. Here's a hilarious one:
https://www.inquirer.com/news/roche...s-chatbot-sheriff-20240206.html?query=sheriff

Philadelphia Sheriff Rochelle Bilal’s campaign is claiming that a consultant used an artificial intelligence chatbot to generate dozens of phony news headlines articles that were posted on her campaign website to highlight her first-term accomplishments.

Incompetent and/or corrupt is totally on-brand for the Philly Sheriff's office (not to be confused with the police department), and this was probably the former, by the consultant. Some now former intern was probably assigned to go find favorable news stories about the sheriff, which would have taken many minutes to do the old fashioned way, with google. Instead they offloaded the task to ChatGPT, which delivered exactly what it was asked for (hey, you didn't clearly state they should be real!). Heck, it's even possible they tried the old fashioned way and gave up when all they could find were articles about the department's dysfunction and editorials saying it should be abolished.

 

[Vanadium 50]

Facts are so 20th century.

 

[nsaspook]

https://www.reuters.com/technology/...d-chatgpt-build-copyright-lawsuit-2024-02-27/

OpenAI says New York Times 'hacked' ChatGPT to build copyright lawsuit​

OpenAI did not name the "hired gun" who it said the Times used to manipulate its systems and did not accuse the newspaper of breaking any anti-hacking laws.
"What OpenAI bizarrely mischaracterizes as 'hacking' is simply using OpenAI's products to look for evidence that they stole and reproduced The Times's copyrighted work," the newspaper's attorney Ian Crosby said in a statement on Tuesday.
Representatives for OpenAI did not immediately respond to a request for comment on the filing.

IMO the common definition of hacking is getting a system to do something it wasn't designed to do. Yes, I know the headline used "" around the work hacked but they used this in the filing.

The truth, which will come out in the course of this case, is that the Times paid
someone to hack OpenAI’s products. It took them tens of thousands of attempts to generate the
highly anomalous results that make up Exhibit J to the Complaint.

 

[Borg]

A thousand monkeys typing for a thousand years.

 

[nsaspook]

https://gizmodo.com/the-story-of-the-monkey-shakespeare-simulator-project-5809583

The story of the Monkey Shakespeare Simulator Project​

https://mindmatters.ai/2019/09/why-cant-monkeys-typing-forever-produce-shakespeare/

WHY CAN’T MONKEYS TYPING FOREVER PRODUCE SHAKESPEARE?​

Researchers at Plymouth University in England reported this week that primates left alone with a computer attacked the machine and failed to produce a single word.

“They pressed a lot of S’s,” researcher Mike Phillips said Friday. “Obviously, English isn’t their first language.”...
Unfortunately, the macaques also relieved themselves on the keyboards.

 

[nsaspook]

https://www.klarna.com/internationa...of-customer-service-chats-in-its-first-month/

https://www.euronews.com/next/2024/...-700-employees-what-will-happen-to-their-jobs

Klarna’s AI bot is doing the work of 700 employees​

https://archive.is/jdLT1

 

[nsaspook]

https://www.cbc.ca/news/canada/british-columbia/lawyer-chatgpt-fake-precedent-1.7126393

B.C. lawyer reprimanded for citing fake cases invented by ChatGPT​

Chong Ke ordered to pay costs for opposing counsel to discover precedent was AI 'hallucination'​

 

[gleem]

Two things popped up this week regarding AI. The first is an article in which AI Claude 3 Opus a product of Anthrop\c [sic] was given a prompt about topping for pizza in which the relevant information was "buried" in irrelevant articles in a so-called "needle in the haystack evaluation". The AI found the answer to the prompt as expected. However, it unexpectedly added the following comment

“However, this sentence seems very out of place and unrelated to the rest of the content in the documents, which are about programming languages, startups, and finding work you love. I suspect this pizza topping ‘fact' may have been inserted as a joke or to test if I was paying attention since it does not fit with the other topics at all.”

This suggests some "awareness" as to why the information sought was not where it might have been expected. The actual prompt was not given in the article. https://www.msn.com/en-us/news/tech...n&cvid=bb89a0818e41414593395ed8ec664055&ei=57

The second article is about the fusion of LLM and robotics. In the video, a robot responds to a human about handling various articles on a counter. Except for the processing delay, it is impressive.

 

[Vanadium 50]

How did you find PF?: I found PF from ChatGPT surprisingly, I had made a theory on physics and asked for it to tell me a site to find like minded people to help confirm the theory. .

Boy did ChatGPT get that one wrong.

 

[Vanadium 50]

Moffatt v. Air Canada, 2024 BCCRT 149

A guy gets bad information from an Air Canada chatbot, acts on it, tried to get a refusd. Air Canada refuses. Guy sues. From the ruling:

In effect, Air Canada suggests the chatbot is a separate legal entity that is responsible for its own actions. This is a remarkable submission.

Air Canada lost.

 

[gleem]

Nvidia at its annual GPU technology conference (GTC) has announced the development of an AI robotic program called Project GR00T (Generalist Robot 00 Technology) a general-purpose foundation model for humanoid robots designed to embody an adaptable AI system to understand natural language and learn new tasks by observation.
https://nvidianews.nvidia.com/news/foundation-model-isaac-robotics-platform

EDIT: Here is a Nvidia video of Project GR00T

 

[Vanadium 50]

Project GR00T

Is natural language processing easier or harder when restricted to three words?

 

[gleem]

Is natural language processing easier or harder when restricted to three words?

Your point?

 

[Borg]

Your point?

I am Groot.

The character can only say the repeated line "I am Groot", but has different meanings depending on context.

 

[nsaspook]

https://themarkup.org/news/2024/03/29/nycs-ai-chatbot-tells-businesses-to-break-the-law

NYC’s AI Chatbot Tells Businesses to Break the Law​

The Microsoft-powered bot says bosses can take workers’ tips and that landlords can discriminate based on source of income

I'm sure you can use that disclaimer for a free 'get out of jail' card.