With ChatGPT, is the college essay dead?

[jtbell]

This article reminded me of the current Fun with ChatGPT thread in General Discussion:

The College Essay Is Dead (The Atlantic)

Suppose you are a professor of pedagogy, and you assign an essay on learning styles. A student hands in an essay with the following opening paragraph:

The construct of “learning styles” is problematic because it fails to account for the processes through which learning styles are shaped. Some students might develop a particular learning style because they have had particular experiences. Others might develop a particular learning style by trying to accommodate to a learning environment that was not well suited to their learning needs. Ultimately, we need to understand the interactions among learning styles and environmental and personal factors, and how these shape how we learn and the kinds of learning we experience.

Pass or fail? A- or B+? And how would your grade change if you knew a human student hadn’t written it at all?

[...]

Going by my experience as a former Shakespeare professor, I figure it will take 10 years for academia to face this new reality: two years for the students to figure out the tech, three more years for the professors to recognize that students are using the tech, and then five years for university administrators to decide what, if anything, to do about it.

My wife (a retired professor of German language and literature) commented that students will have to be forced to write their essays in the classroom, after having their phones confiscated.

 

[jedishrfu]

I imagine they could use software to compare essay styles and hope that the GPT-3 will have a marked signature in how it writes text.

ALso one might be able to collect all essays written by the student and see if this one matches their writing style.

Basically, it will be a silent war of AI tools fighting for dominance.

 

[jack action]

What is the difference between this problem and having the student pay someone else to write the essay?

The solution is the same in both cases: You have to make the students explain their work, like an oral presentation in front of the class. Either the student wrote it, or he studied it so much that he understands it.

"But he didn't write it! How will he be able to produce texts in the future?" I hear some say. I know I learned math in the past, and today I use a computer to do most of my calculations. Some methods I learned, I haven't done for a long time. It doesn't mean I don't know what I'm doing (or what the computer is doing), only that I'm more efficient. In the rare case where I would need to correct the computer and get to do it by hand - or at least study the code to see how the computer does it - it would most likely be like riding a bicycle: it would come back because I know what I'm looking for.

Using GPT-3 for writing a text is like using a finite element analysis software for an engineer.

 

[Vanadium 50]

What is the difference between this and paying someone to write the essay for you? In my view, this is not a new problem - technology is just making it cheaper. Colleges can, if they chose, make it more expensive by increasing penalties, but will they?

 

[Mark44]

What is the difference between this problem and having the student pay someone else to write the essay?

None, other than the "someone else" would be a person.

The solution is the same in both cases: You have to make the students explain their work, like an oral presentation in front of the class. Either the student wrote it, or he studied it so much that he understands it.

Which is why @jtbell said this:

My wife (a retired professor of German language and literature) commented that students will have to be forced to write their essays in the classroom, after having their phones confiscated.

Using GPT-3 for writing a text is like using a finite element analysis software for an engineer.

I don't see this as a valid analogy. With the FE software the engineer has to provide some input data. And further, the engineer presumably has gone through a course in which the rudimentary operations being performed by the software have been done by hand or at least understood. Using some AI software to write an essay doesn't require any understanding of any details of the essay to be written and provides no measure of what the student actually knows.

 

[jack action]

With the FE software the engineer has to provide some input data. And further, the engineer presumably has gone through a course in which the rudimentary operations being performed by the software have been done by hand or at least understood.

The input data provided may only be a CAD drawing that the person hasn't even made. The meshing is done by the software. Then - to use your word - presumably, the person knows what to look for and interpret the results ... presumably.

On the other hand, a journalist could ask AI to write an article about a particular subject. To repeat your words again, the journalist presumably has gone through a course in which the rudimentary writing skills being performed by the software have been done by hand or at least understood and thus the final article can be reviewed and corrected before being submitted.

 

[Mark44]

The input data provided may only be a CAD drawing that the person hasn't even made. The meshing is done by the software. Then - to use your word - presumably, the person knows what to look for and interpret the results ... presumably.

Again, I don't see that this is analogous to the situation of a student using AI to write an essay. An engineer would already have shown competence in his or her area by virtue of university classes, a degree, and some sort of certification. This is completely different from that of a student who has yet to show competence in the area of the subject of the essay. A good engineer would probably have some experience working with the CAD software to be able to create the input data, and might also have some insight about how fine or coarse the mesh should be to get reasonable results that don't take too long to compute.

On the other hand, a journalist could ask AI to write an article about a particular subject. To repeat your words again, the journalist presumably has gone through a course in which the rudimentary writing skills being performed by the software have been done by hand or at least understood and thus the final article can be reviewed and corrected before being submitted.

The journalist would likely have gone through a course to learn writing skills and how to do research in the area of choice for the article, to at least get a sense of the basic ideas involved in the background for the article.

 

[Algr]

I imagine they could use software to compare essay styles and hope that the GPT-3 will have a marked signature in how it writes text.

ALso one might be able to collect all essays written by the student and see if this one matches their writing style.

Basically, it will be a silent war of AI tools fighting for dominance.

I abandoned any thought of studying writing in college because the administration did not seem to know what a "false positive" meant. Their definition of "plagiarism" was so broad that merely stating common knowledge or describing something you witnessed yourself could count. They showed us samples of "plagiarized" writing that bore no connection to the original other than describing the same event in the same newspaper style. You could prove you didn’t know a work existed, and still be guilty of plagiarizing it! I took my concerns to the staff and all they could say was "Stop looking for ways to plagiarize." They just didn't care if honest writing was possible or not.

 

[jedishrfu]

Yes, this is always a possibility. We see it here on PF where some posts are flagged by Akismet code. Basically, it tells us if the post has been used on other websites that are a part ofthe Akismet network. Its a good tools to discover spam texts but sometimes it just flags a short post asking a simple question in the manner that many posters might ask.

https://en.wikipedia.org/wiki/Akismet

 

[Jarvis323]

Maybe in the future people won’t need traditional skills. Instead the skills people will need will be how to use AI technologies.

A future college curriculum might look like this (generated courtesy of GPT):

Course Catalog:

1. Introduction to GPT 10: In this course, students will learn about the basics of GPT 10 and how it functions. Students will also learn how to interact with GPT 10 and how to give it commands. Instructor: GPT 10
2. Advanced GPT 10: In this course, students will learn about the advanced features of GPT 10 and how to utilize them to their advantage. Students will also learn how to customize GPT 10 to suit their needs and preferences. Instructor: GPT 10
3. GPT 10 for Business: In this course, students will learn how to use GPT 10 to improve their business operations. Students will learn how to use GPT 10 to automate tasks, analyze data, and make better decisions. Instructor: GPT 10
4. GPT 10 for Creativity: In this course, students will learn how to use GPT 10 to unleash their creativity. Students will learn how to use GPT 10 to generate new ideas, create art and music, and more. Instructor: GPT 10
5. GPT 10 for Education: In this course, students will learn how to use GPT 10 to enhance their education. Students will learn how to use GPT 10 to improve their learning experience, access new knowledge, and more. Instructor: GPT 10
6. GPT 10 for Health: In this course, students will learn how to use GPT 10 to improve their health and wellbeing. Students will learn how to use GPT 10 to monitor their health, track their fitness, and more. Instructor: GPT 10
7. GPT 10 for Entertainment: In this course, students will learn how to use GPT 10 to enhance their entertainment experience. Students will learn how to use GPT 10 to access new forms of entertainment, create personalized experiences, and more. Instructor: GPT 10

 

[StevieTNZ]

Their definition of "plagiarism" was so broad that merely stating common knowledge or describing something you witnessed yourself could count.

These are guidelines for avoiding plagiarism at the university I'm enrolled with:
https://owll.massey.ac.nz/referencing/plagiarism.php

 

[Algr]

From Massa university on Plagiarism:

• If you copy more than about three consecutive words from a source, put the words in quotation marks

I challenge anyone to produce a three word description of plagiarism that Google can't find. Every meaningful combination of three words exists somewhere on the internet.

Using the ideas of a source you have read, even if you write it in a different way, is still plagiarism:

...so the three words don't even have to match. Can you even say that plagerism exists without attributing it to someone? And notice that there is no expectation that the accuser demonstrate that "you have read" something. You are responsible for knowing the existence of everything in any library anywhere.

Many new students worry about accidentally plagiarising. This is perfectly natural! The rules of referencing are complex and intimidating at first. Academic study involves a lot of reading, and it can be difficult to keep track of the sources of ideas. Most study guides contain stern warnings about the penalties for plagiarism.

It's actually very difficult to plagiarise accidentally.

• If you follow the guidelines on this page, plagiarism is easily avoided

Everything is "easy" for those who don't have to to it themselves. How do you recognise that an idea you think is original was actually said before? You'd have to prove a negative - that the idea DOESN'T exist anywhere in published literature. That is obviously impossible, so original thought is effectively banned. Also, there is no such thing as a person who starts with zero knowledge on a subject and only knows things from current research. How do you account for ideas you may have heard about when you were twelve?

I have encountered papers that are so buried in citations and attributions as to be completely unreadable. More the half the text was citations, and I just could not find where the actual sentences started or ended. Notice that any textbook or video that people are expected to actually learn from does NOT do this. Put on an episode of Cosmos and show me the attributions. Sagan or Tyson will sometimes drop a researcher's name, but they do nothing like what students are expected to do.

==========================================
Under this standard, the only way to protect yourself from accusations of plagiarism is to perform "reverse research". This is where you Google every last idea in your paper, every sentence, and attribute it to SOMEONE, even if it is someone you have never heard of before. You are only safe if you have no claim to original thought. That is probably how the unreadable paper I describe above was written.

 

[Algr]

As you can see above, Plagiarism anxiety is a sore point with me, and is why I am not currently in academia. I was never accused of plagiarism, but I feel that success in the fields I aspired to had more to do with protection from friends in high places then the ability to do the work.

 

[Hornbein]

Under this standard, the only way to protect yourself from accusations of plagiarism is to perform "reverse research". This is where you Google every last idea in your paper, every sentence, and attribute it to SOMEONE, even if it is someone you have never heard of before. You are only safe if you have no claim to original thought. That is probably how the unreadable paper I describe above was written.

Hmm, it should be possible to automate this process. Could be an opportunity.

Readers could also buy a program that removes all the citations so that the paper is readable again. So you can "get 'em coming and going."

 

[Algr]

Hmm, it should be possible to automate this process. Could be an opportunity.

Yes. Unfortunately what I was trying to read was an actual paper book. Google wasn't around yet.

 

[StevieTNZ]

If you are deliberately copying the words, then quote them. That's what I take from reading it.

Be a bit more liberal in how you construe the webpage. Don't be so literal.

 

[Algr]

If you are deliberately copying the words, then quote them. That's what I take from reading it.

Following rules is irrelevant if you can't defend yourself against untrue accusations. If someone claims you copied the words, there is no way to show you didn't, and no burden on the accuser to show that you knew the other work existed. There is no accountability for false accusations of plagiarism. And often no distinction between the accuser and the judge.

Edit:

Be a bit more liberal in how you construe the webpage. Don't be so literal.

It makes no difference how I construe the webpage unless I am the one with the option to do the expelling. It is like Florida's "Don't say gay" bill. It doesn't literally say "Don't say gay", but it is clearly written to allow that interpretation to be enforced.

When it comes to laws and policies, the words on the page are little more than excuses. It is how things are interpreted and what actually gets enforced that matters.

 

[jack action]

 

[Algr]

...the entire works of Shakespeare...​

Three words.
The policy says "three words".
(Also, I added a response above to StevieTNZ's [edit?].)

 

[jedishrfu]

Plagiarism is a difficult charge to make. I would hope the university doesn't just decide on a red flag from a scanning program. These programs would have to provide an annotated version of your writing with plagiarism phrases highlighted so that a reviewer can decide whether you actually infringed upon another writers work.

As an example, if the highlighting revealed some phrase coming from multiple sources then one could safely say that it is a phrase in common usage and move on. However, if the hilighting showed multiple important phrases, sentences or paragraphs coming from a single source then the likelihood of plagiarism is evident.

I did hear of students getting flagged because scanning software reported that they had plagiarized their own essay. This is considered just as wrong as plagiarizing from another writer, basically reusing work that had been submitted for credit earlier.

https://www.latrobe.edu.au/mylatrob...cling-your-own-work-can-get-you-into-trouble/

https://www.asseltalaw.com/blog/201...-in-school-answer-from-a-lawyer-for-students/

https://academicintegrity.unimelb.edu.au/

Personally, I disagree with self-plagiarism but in this new AI policing environment, it's best to annotate your work clearly identifying those passages that you had written previously. Turnitin software keeps a database of essays and can spot self-plagiarized passages if your college uses it.

One case to consider is if you upload a version of your writing to scan for plagiarism not realizing that TurnItIn would add it to their database meaning when the prof does a scan your prior version will trigger a plagiarism alert on writing you have yet to submit for credit.

https://inkforall.com/copy-editing/plagiarism-checker/how-do-professors-check-for-plagiarism/

Lastly, here is an example of the rules governing plagiarism at the University of Washington:

https://depts.washington.edu/pswrite/plag.html

Some famous plagiarism cases:

https://www.ranker.com/list/high-profile-cases-of-plagiarism/janaegreen

https://www.mentalfloss.com/article/21761/4-famous-cases-plagiarism

 

[StevieTNZ]

Three words.
The policy says "three words".
(Also, I added a response above to StevieTNZ's [edit?].)

It actually says "about three consecutive words".

All in all, be reasonable and use common sense. Instead of arguing with the university. I'm sure they'd be sick of you doing that.

 

[jedishrfu]

As an aside, I considered the total number of three words combos that can be made from this website:

https://thelanguagedoctors.org/which-language-has-more-words/

English is one of the most populous languages in terms of dictionary entries. Oxford English contains over 200,000 words, with 171,476 active and 47,156 inactive words.

So we take $200,000^3$ which is $8 \cdot 10^{15}$.

It's hard to reduce it further without caveats like:

- word order,
- repeated words,
- use of common expressions,
- number of words in your writing,
- ...

to determine the probability of quantum entangling your words with other authors and implying plagiarism.

I leave the detailed calculation to the students of combinatorics.

 

[TeethWhitener]

Well, “the prime minister” probably sees a touch more use than “flamboyantly neoclassical duodenum.”

Also, on topic: good riddance to the college essay. At least make it optional with the caveat that if it bores the admissions officer to death, then you’re automatically rejected. I’ve done a lot of reviewing for scholarships/fellowships/etc., and I’ve seen at most 3 actually interesting personal statements. (In reality, there was one amazing life story, two or so mildly interesting essays because of strong writing ability, and a whole lot of lukewarm Miss America speeches)

 

[Office_Shredder]

Don't worry, I'll soon be publishing The Complete Work of Three Word Combinations so you can just cite me a couple times per sentence and be protected.

 

[jedishrfu]

My niece did a lot of admission essay reviews. She said one of the most common themes was the Rudy essay citing the movie Rudy and overcoming hardships to get to college.

What do you expect from a high-school student with very little essay background. I'm sure my essay was pretty bland if not full of English mistakes.

 

[256bits]

At least make it optional with the caveat that if it bores the admissions officer to death, then you’re automatically rejected

ChatGPT would be automatically rejected due to repetition on and on, without actually saying much.
If all the essay bots are like that, then it should be not all that difficult to tell that a bot wrote it, or the the student wrote it and is just somehow trying to reach the 1000-2000 word limit.

At Univ I used to avoid writing my essay assignments until the night before and still gets A's.
Re-reading them quite a few years later, I was astonished at how 'bad' they really were.
Pity the poor professor who had to read through that.

So for the opening post paragraph, I would right away say that there are two choices - the student is space filling, or a bot wrote it - probably a bot since the grammatical mistakes are none to ziltch, and it is boring.

 

[jack action]

At Univ I used to avoid writing my essay assignments until the night before and still gets A's.
Re-reading them quite a few years later, I was astonished at how 'bad' they really were.

IMHO, teachers giving A's to 'bad' essays is a much more important problem - and easier to solve - in today's higher education systems than students cheating.

But choosing between taking the money of a mediocre student is a lot easier than arguing with him, his parents, or his lawyer.

 

[Greg Bernhardt]

Even though ChatGPT uses GPT-3.5, this GPT-2 detector still has moderate success in detecting its content as AI. For those not impressed, that is ok, but just think about 3-5 years from now. GPT-4 is due out early next year.

 

[256bits]

IMHO, teachers giving A's to 'bad' essays is a much more important problem - and easier to solve - in today's higher education systems than students cheating.

But choosing between taking the money of a mediocre student is a lot easier than arguing with him, his parents, or his lawyer.

( Should that be taken as personal. )
( Student was not mediocre scholastically, by the way, and second in line to be valedictorian upon graduation from high school does say something about information comprehension, absorption, and understanding, other than that gleamed from the premise being presented ).

Nonetheless, carrying on, I am not in any way in agreement.
Case in point - Einstein.
You can read this
https://www.samuelobe.com/genius/
and perhaps present a more fully developed argument, if what is said is to be believed about a mediocre student who became world famous.

 

[jack action]

( Should that be taken as personal. )

Sorry, I should've said student handing mediocre work, not mediocre student.

 

[jack action]

Even though ChatGPT uses GPT-3.5, this GPT-2 detector still has moderate success in detecting its content as AI. For those not impressed, that is ok, but just think about 3-5 years from now. GPT-4 is due out early next year.

What I'm I supposed to understand about that:

 

[256bits]

Even though ChatGPT uses GPT-3.5, this GPT-2 detector still has moderate success in detecting its content as AI. For those not impressed, that is ok, but just think about 3-5 years from now. GPT-4 is due out early next year.

I wrote
"A person with red skin has spent too much time in the sunlight on hot summer days without UV protection. "
giving 42.41% real/57.59% fake%

I wrote
"A person of red skin has spent too much time in the sunlight on hot summer days without UV protection. "
( a change of one word 'with' to ' of'' ),
giving 26.67% fake/72.33% real.

Adding in the word 'colour' after skin, the second sentence jumps it up to 51.52% fake.

Making me worried that I am part bot and don't know it.

 

[256bits]

Sorry, I should've said student handing mediocre work, not mediocre student.

There's the rub - how to tell the difference, except through performance from exams and essays.
I don't think it is a 'perfect' system, if there ever will be.

 

[Jarvis323]

I don’t think it would be possible to detect chatgpt in general, but maybe its generic answers with some success.

ChatGPT output depends a lot on the input. For example, if you use latex symbols it performs better in mathematics. If you ask it a complex question using non trivial words and concepts, its output will be more complex and interesting sounding. If you ask a more basic question it gives a more basic answer.

Basically, it considers your prompt as context, and in context, and (effectively) speaks to its audience. E.g., academic sounding questions get more academic sounding answers. It also takes into account insight and feedback. You can even explain theoretical results outside of its training data, and it understands (to a degree) and can integrate that into its analysis in a logical way. Furthermore, you can ask it to write a certain way. E.g., you could ask it to be concise, or you could ask it to use imperfect grammar. You can even give it some text as an example and tell it to use that style or rewrite it in a different style.

Ultimately, it is much more powerful than it seems at first, but getting the most out of it is interactive and experimental. And its writing quality is not what is the most impressive (to me); it seems to have general problem solving skills. It can not only write essays, it can do homework assignments in undergraduate theory of computation.

I think what ChatGPT demonstrates, is that language models are not just language models. Text data contains not just rules of language, but also logic, reasoning, concepts, and abstraction. in order to learn how to predict text in context, in general, you have to model those things as well.

 

[Astronuc]

The College Essay Is Dead (The Atlantic)

Interestingly and coincidentally, there has been a flurry of articles on OpenAI's ChatGPT.

Another article in The Atlantic

Let me be candid (with apologies to all of my current and former students): What GPT can produce right now is better than the large majority of writing seen by your average teacher or professor. Over the past few days, I’ve given it a number of different prompts. And even if the bot’s results don’t exactly give you goosebumps, they do a more-than-adequate job of fulfilling a task.

David Herman, The End of High-School English, Dec 9, 2022
https://www.theatlantic.com/technol...gpt-writing-high-school-english-essay/672412/

. . . The rudiments of writing will be considered a given, and every student will have direct access to the finer aspects of the enterprise. Whatever is inimitable within them can be made conspicuous, freed from the troublesome mechanics of comma splices, subject-verb disagreement, and dangling modifiers.

But again, the majority of students do not see writing as a worthwhile skill to cultivate—just like I, sitting with my coffee and book, rereading Moby-Dick, do not consider it worthwhile to learn, say, video editing. They have no interest in exploring nuance in tone and rhythm; they will forever roll their eyes at me when I try to communicate the subtle difference, when writing an appositive phrase, between using commas, parentheses, or (the connoisseur’s choice) the em dash.

Which is why I wonder if this may be the end of using writing as a benchmark for aptitude and intelligence. After all, what is a cover letter? Its primary purpose isn’t to communicate “I already know how to do this job” (because of course I don’t) but rather “I am competent and trustworthy and can clearly express to you why I would be a good candidate for this job.” What is a written exam? Its primary signal isn’t “I memorized a bunch of information” but rather “I can express that information clearly in writing.” Many teachers have reacted to ChatGPT by imagining how to give writing assignments now—maybe they should be written out by hand, or given only in class—but that seems to me shortsighted. The question isn’t “How will we get around this?” but rather “Is this still worth doing?”

I believe my most essential tasks, as a teacher, are helping my students think critically, disagree respectfully, argue carefully and flexibly, and understand their mind and the world around them. Unconventional, improvisatory, expressive, meta-cognitive writing can be an extraordinary vehicle for those things. But if most contemporary writing pedagogy is necessarily focused on helping students master the basics, what happens when a computer can do it for us? Is this moment more like the invention of the calculator, saving me from the tedium of long division, or more like the invention of the player piano, robbing us of what can be communicated only through human emotion?

How Google Got Smoked by ChatGPT
The most embarrassing part is that the search giant has a chatbot that’s better.
https://slate.com/technology/2022/12/chatgpt-google-chatbots-lamda.html
By Alex Kantrowitz, Slate, Dec 10, 2022

Google’s had an awkward week. After years of preaching that conversational search was its future, it’s stood by as the world discovered ChatGPT.

The powerful chatbot from OpenAI takes queries—some meant for the search bar—and answers with astonishing conversational replies. It’s shared recipes, reviewed code, and argued politics so adeptly that screenshots of its answers now fill social media. This was the future Google promised. But not with someone else fulfilling it.

How Google missed this moment is not a simple matter of a blind spot. It’s a case of an incumbent being so careful about its business, reputation, and customer relationships that it refused to release similar, more powerful tech. And it’s far from the end of the story.

Stumbling with their words, some people let AI do the talking
The latest AI sensation, ChatGPT, is easy to talk to, bad at math and often deceptively, confidently wrong. Some people are finding real-world value in it, anyway.
https://www.washingtonpost.com/technology/2022/12/10/chatgpt-ai-helps-written-communication/
By Drew Harwell, Nitasha Tiku and Will Oremus, Dec 10, 2022

The New Chat Bots Could Change the World. Can You Trust Them?
https://www.nytimes.com/2022/12/10/technology/ai-chat-bot-chatgpt.html
By Cade Metz, Dec. 10, 2022
Cade Metz wrote this article based on months of conversations with the scientists who build chat bots and the people who use them.

Siri, Google Search, online marketing and your child’s homework will never be the same. Then there’s the misinformation problem.

Aaron Margolis, a data scientist, says that new chat bots are remarkable but that their answers can conflate fact with fiction.

This month, Jeremy Howard, an artificial intelligence researcher, introduced an online chat bot called ChatGPT to his 7-year-old daughter. It had been released a few days earlier by OpenAI, one of the world’s most ambitious A.I. labs.

He told her to ask the experimental chat bot whatever came to mind. She asked what trigonometry was good for, where black holes came from and why chickens incubated their eggs. Each time, it answered in clear, well-punctuated prose. When she asked for a computer program that could predict the path of a ball thrown through the air, it gave her that, too.