My ChatGPT Experience So Far

[Hornbein]

I've had ChatGPT answer questions correctly that most math PhDs wouldn't know. I've also had it fail on something most six-year-olds could do. And everything in between, a mixture of right and wrong. Nevertheless I find it very useful, even when it's wrong.

In his iconic 1905 special relativity paper the only credit offered is to Michele Besso, an engineer friend with a remarkably flakey mind. [Boston Red Sox pitcher Bill "Spaceman" Lee appears to have been the original flake.] Albert found their conversations most helpful. Besso said, "the eagle Einstein took the sparrow Besso under his wing and the sparrow flew a little higher." I feel sure that Besso's contribution was his unique perspective on the matter. That's how I have found ChatGPT for my geometry research. Though it is often wrong, it's free of charge, it responds immediately 24/7, and it sees things from a different angle that is stimulating and thus helpful.

 

[FactChecker]

It's one thing to review the ChatGPT results on a subject you know. It's another if you do not know the truth already. It's the difference between "a different angle" versus the only angle you see.

 

[russ_watters]

Nevertheless I find it very useful, even when it's wrong.

In his iconic 1905 special relativity paper the only credit offered is to Michele Besso, an engineer friend with a remarkably flakey mind. [Boston Red Sox pitcher Bill "Spaceman" Lee appears to have been the original flake.] Albert found their conversations most helpful. Besso said, "the eagle Einstein took the sparrow Besso under his wing and the sparrow flew a little higher." I feel sure that Besso's contribution was his unique perspective on the matter. That's how I have found ChatGPT for my geometry research.

That's a good example of how some creative/outside the box thought can be useful in small doses, in a sandbox, for an expert. In the wrong hands it's a Dunning-Kruger multiplier.

 

[martinbn]

I've had ChatGPT answer questions correctly that most math PhDs wouldn't know.

I am curious about this. Do you remember any of the questions you asked?

 

[Hornbein]

I am curious about this. Do you remember any of the questions you asked?

Sure.
"If you have two random 2D planes in a 4D space, what is the probability that they intersect?"

It did very well with that one, supplying a proof of its answer. I didn't believe it at first. It was also very good at describing an obscure relation within the Hopf fibration.

"How would a duocylinder roll on a 3D plane with gravity?"

It said "it could theoretically roll in two perpendicular directions at the same time", which is nonsense (unless you look at it in a too-peculiar way).

"Draw a red square."

It insisted on drawing a cube.

It said that the rotational circles of a 4D sphere that one finds in the Hopf fibration are linked, a widely believed falsehood. In a 4D space 2D circles can never be linked.

In a number of questions it responded that it didn't know, which was a very good answer. At that point I would expect that no one else on Earth knew either. The way it phrases it is, it could be this or it could be that.

 

[Hornbein]

I live in Indonesia. Skype is shutting down May 7th, a problem because I use it to get verification texts from banks. I told my girlfriend who will have the same problem. She refused to believe it because ChatGPT said it was a rumor. I had to send her a screenshot from Skype itself announcing the shutdown. Then she sent me ChatGPT's advice on what to do, all of which was wrong. Arggh!

 

[Trysse]

I use ChatGPT a lot to review the texts I write and read. The obvious part is copy editing—finding grammatical errors and making formulations smoother.


But I also use it to summarize texts, both ones I get and ones I write. If it’s a text I receive, I want to see if it’s worth my time before reading the whole thing. Sometimes, if I’ve already read it, I let ChatGPT summarize it and then discuss my take on it with the model.


When I upload my own texts, I check whether ChatGPT picks up on the main points and the overall intention. If it does, I take that as a sign that my text is clear. I also ask for feedback on clarity, style, and word choice.


Sometimes, I run the same text and questions through different LLMs (Grok, DeepSeek, Le Chat, etc.). ChatGPT tends to be the most encouraging when discussing my own writing, which is nice—but after a while, it gets a bit much. When that happens, I either switch to another LLM or tell ChatGPT to tone it down.


As with the Skype example above, I’ve also seen that LLMs sometimes make stuff up. Once, after discussing a document with ChatGPT for a while, I asked for a summary of the original document. The summary included points that weren’t in the original text but had come up in our discussion about it. When I pointed this out, ChatGPT agreed and corrected it. I assume the issue was that the model integrated parts of the conversation into the summary.


For me, that’s not a big deal, but it’s a reminder not to take ChatGPT’s output at face value without a final check from a human perspective.

 

[Hornbein]

Yes I also find Chat sucks up to you, telling you how delightful your stupidity is. I don't care for that but surely it adds greatly to its popularity.