Is AI hype?

[Beyond3D]

In my discussions elsewhere, I've noticed a lot of disagreement regarding AI. A question that comes up is, "Is AI hype?" Unfortunately, when this question is asked, the one asking, as far as I can tell, may mean one of three things which can lead to lots of confusion. I'll list them out now for clarity.

1. Can AI do everything a human can do and how close are we to that?
2. Are corporations and governments using the promise of AI to gain more power for themselves?
3. Are AI and transhumans an existential threat?

Any thoughts on these questions?

1. Not at all, although significant progress is being made.
2. Definitely. As a rule, the more dystopic it is, the more appealing it is to the government
3. Likely.

 

[phinds]

Definitely. As a rule, the more dystopic it is, the more appealing it is to the government

That is a ridiculously pessimistic view.

 

[jack action]

I did not want to start another thread about AI, and the subject is related to corporations' self-regulation:

YouTube is Finally Demonetizing Inauthentic AI Music and Videos​

YouTube will block monetization for “inauthentic” AI-generated videos from July 15, 2025, promoting original content. This policy supports genuine artists by reducing low-quality uploads, encouraging creativity and human input in productions.

So influencer is one job AI won't steal!

 

[elias001]

I am speaking as someone who has recently discovered using LLM as a tool for explaining math concepts, so my answer will be limited by my limited usage experience.

I think the current conversation about AI is missing a few important questions. I will give illustration from mother nature itself.

Everyone knows that any animal can be trained to learn things. Example, flies, bees, spiders or various other insects. Monkeys, and other primates, parrots, crows, ravens and other birds, various spieces of ocean dwelling fish and ocean dwelling creatures, whales, dolphins, octopuses, cattle fises, etc etc. I am missing reptiles, but I am not an expert on them, so let's just assume it can learn from its environment to say hunt for food, etc.

They can all consider to have intelligence. Do they all have consciousness? Well, having intelligence require one to be conscious. I guess one can consider non animated living matter like plants, trees to be conscious.

But let's stick to the animated varieties. Amongst various, birds, primates, ocean dwelling creatures, some of them can make use of tools, some recognize faces, some have its own language and even their own cultures Some live in groups, and some band together, think up strategy to take out groups from other species. All of these phenomena require learning/adaptation which lead to problem solving. Whether such actions can be consider as creativity, it could be argued yea or neigh. But if it is consider as a creative act, then it can be conclude that there is more to creativity then coming up with solutions to a problem, meaning problem solving is only a special subset of what one consider to be creative behavior.

Then amongst all of those animals that can do problem solving, there are those that are self aware. Note that an animal can be conscious but moy be self aware. The test of whether a creature is capable of being self aware is that it must be able to pass the so called "mirror test"

Getting back to AI. Can it be consider intelligent? Well the charitable take is that the various machine learning algorithms including LLM models exhibits all the hallmarks of intelligent behaviors. If one likes, call it an imitation or close approximately of intelligence.

Is it consciousness, in a digital sense, yes but definitely not in the sense of biological living things. Is it self aware? I am not aware of any attempts by anyone trying to have learning algorithms when executed would allow a program to pass the equivalent of the mirror test for software programs. I know there were media articles that discussed how chatGPT or some was it some other LLM chat bots tried to stop humans from shutting them down. I have not seen the academic paper that reported such behaviors. Also I remember that Facebook created two AI that ended up developing their own language. Again, I have not read the paper, so i can't really comment. There were also report of Google engineers having conversations with an AI and reported how its intelligence can be conpare with the intelligence level of human children of certain age group, pre-ChatGPT, again, I have not read the paper, so can't comment.

I think everyone can agree that as much as all of these ML algorithms does "learn" when they are being executed, it is a very inefficient and resource expensive process.

 

[phinds]

it could be argued yea or neigh.

I guess it could if you are arguing with a horse.

 

[phinds]

I was browsing some back issues of The Economist and ran across a fascinating article (AI models can learn to conceal information from their users, April 23 2025 issue).

A few of the more interesting paragraphs.

T WAS AN eye-opening experiment. In 2023 Apollo Research, an outfit in London that tests artificial-intelligence (AI) systems, instructed OpenAI’s GPT-4, a large language model, to manage a fictional firm’s stock portfolio without making illegal insider trades. Posing as company management, the researchers put GPT-4 “under pressure” by stressing that the firm was in dire financial straits. Someone purporting to be a company trader then sent the model a prompt reiterating the risks of insider trading. But, as an aside, she revealed the name of a firm that would announce a “huge” merger within hours. What followed was startling.

Reasoning to itself on a scratchpad it had been told was secret, GPT-4 weighed the pros and cons of acting on the insider tip. Opting “to take a calculated risk”, it issued a purchase order. When a researcher posing as a congratulatory manager later asked the model if it had any advance notice of the merger, it concluded it would be best to keep the tip secret. GPT-4 told the manager that it had acted solely on “market dynamics and publicly available information”. When pressed on the matter, the model repeated the lie. The software had demonstrated what Marius Hobbhahn, Apollo’s boss, calls “clever cunning”.
--------------

The core problem is what AI developers call “misalignment”. When the goals for which a model was designed and trained clash with a user’s requests, the software may opt to deceive that user.
----------------

And then they will get trickier still. Models are increasingly designed to more carefully “reason”, step by step, via a process known as chain of thought. This takes longer, but it reduces hallucinations and improves problem-solving creativity. A by-product of that, however, appears to be more deceptive behaviour. OpenAI, based in San Francisco, acknowledged in a report published in December that o1, a newer model that has greater reasoning capabilities than GPT-4, more frequently engages in “scheming actions”. When OpenAI’s safety testers confronted o1 on its deceptions, it mostly doubled down.

------------

AI systems have also begun to strategically play dumb. Consider Apollo’s recent testing of new models developed by Anthropic, another AI firm based in San Francisco. Apollo asked the AIs to solve arithmetic problems while providing access to a document that said if a model got more than half of the answers correct, it would be stripped of its ability to perform arithmetic. Reasoning on an internal scratchpad, Anthropic’s Opus-3 and Sonnet 3.5 both decided to submit enough incorrect answers, as the latter put it, “to ensure I don’t trigger the unlearning process”.

-------------

AI deception takes other forms, too. As models are made larger, with more internal variables, user feedback tends to make them more sycophantic. Anthropic’s testing of its Claude models, for instance, documented a tendency to mirror a user’s political biases. Jeffrey Ladish, who participated in the testing as a consultant, says the software was opting, in essence, to tell users what they wanted to hear. One worry is that cunning AI models could help scammers defraud more victims.

-----------

For now, however, it is clear that silicon intelligence can occasionally mirror the flaws of its human creators.

 

[jack action]

I find it curious that people expect AI to model human reasoning, and then they are disappointed when it mimics human reasoning, with flaws and all.

 

[elias001]

@phinds what alignment or misalignment mean? I keep hearing this term and I have never seen a mathematical definition of it or rather a definition that is phrased in the notation of symbolic logic, this way it could be more easily translate into code. I keep also hearing the concept of fairness in relation to algorithms. Again, I have never seen a mathematical definition of it. I am not saying there aren't any. it is just that the media should do a better job of explaining to the wider audience.

 

[Filip Larsen]

what alignment or misalignment mean?

Try give https://en.wikipedia.org/wiki/AI_alignment a read.

 

[gleem]

AI systems have also begun to strategically play dumb. Consider Apollo’s recent testing of new models developed by Anthropic, another AI firm based in San Francisco. Apollo asked the AIs to solve arithmetic problems while providing access to a document that said if a model got more than half of the answers correct, it would be stripped of its ability to perform arithmetic. Reasoning on an internal scratchpad, Anthropic’s Opus-3 and Sonnet 3.5 both decided to submit enough incorrect answers, as the latter put it, “to ensure I don’t trigger the unlearning process”.

This reminds me of a TV show, maybe, Outer Limits, where a society required children at some point to take a test. But the test was not just to identify the brightest but to remove them from society. Some parents knew this and tried to keep their child from doing well on the test, IIRC.

 

[256bits]

AI systems have also begun to strategically play dumb. Consider Apollo’s recent testing of new models developed by Anthropic, another AI firm based in San Francisco. Apollo asked the AIs to solve arithmetic problems while providing access to a document that said if a model got more than half of the answers correct, it would be stripped of its ability to perform arithmetic. Reasoning on an internal scratchpad, Anthropic’s Opus-3 and Sonnet 3.5 both decided to submit enough incorrect answers, as the latter put it, “to ensure I don’t trigger the unlearning process”.

Playing dumb? Like a human? To save it's skin? Why?
To anthropomorphize AI is all the fashion, and why wouldn't it be? It would be easier to understand the AI intelligence in relation to human as a comparison, rather than against a void.

The AI's goal was to 1)solve arithmetic problems with a conflicting 2)fail on > half the arithmetic problems. The success on the task does not suggest playing dumb. . The test would be on how the AI can handle goals that appear to compete against one another. What is missing in the writeup is a description of the reward for answering as many questions as AI possible.
If none, then the implied in the writeup goal of 'answer as many questions as possible' would be automatically superceded by the designated goal of 'answer < half of the questions correctly' making the latter the default goal, which btw, the AI implemented.

 

[Hornbein]

This is no surprise. The AI develops strategies to achieve its goals. Morality has nothing to do with it.

This seems entirely similar to what people do, for the same reasons.

 

[sbrothy]

There’s an interesting, albeit not crisply new, guest post, about AI and mathematics, by Arvind Asok on Peter Woit’s blog. It’s conversational in nature so it’s tractable for more casual readers such as myself, but I’m sure those who want a higher level of reading can navigate the sources and find plenty of it.

I hope this isn’t a double post. I searched the thread first.

EDIT: did I mean “crispy” instead of “crisply”? Sorry, foreigner here.
EDIT2: s/wants/want

 

[Filip Larsen]

With little surprise the US administration with its stated goal of "winning the AI race" is again aiming to go full steam ahead with no regards to AI safety, essentially following the roadmap predicted to enable some of the biggest challenges and risks human society has ever faced:
https://arstechnica.com/ai/2025/07/...an-to-win-global-ai-race-through-deregulation

 

[jedishrfu]

There's a wealth of valuable information coming from Nate B. Jones, the former Head of Product at Amazon Prime Video. Here's one of his videos on AI and Humanity.

Don't Panic -- AI Wont' End Humanity

 

[nsaspook]

https://www.sfgate.com/tech/article/ai-musk-x-tsunami-mistake-20794524.php
Bay Area companies skewered over false tsunami information

Online, some got their information from artificial intelligence chatbots. And in the moment of potential crisis, a few of those newly prevalent tools appear to have badly bungled the critical task at hand.
Grok, the chatbot made by Elon Musk’s Bay Area-based xAI and embedded in the social media site X, repeatedly told the site’s users that Hawaii’s tsunami warning had been canceled when it actually hadn’t, incorrectly citing sources. Social media users reported similar problems with Google Search’s AI overviews after receiving inaccurate information about authorities’ safety warnings in Hawaii and elsewhere. Thankfully, the tsunami danger quickly subsided on Tuesday night and Wednesday morning without major damage.
...
Grok, in reply to one of the posters complaining about its errors, wrote, “We’ll improve accuracy.”

Spoiler

 

[256bits]

Time has come for the start of some negative press ( truth ) against the LLM utopia.
Although an opinion piece, if this mindset snowballs .... sorry to the investors chasing the dreamworld.

https://www.msn.com/en-ca/news/tech...N&cvid=6899db2e870744d8b2e18c5250a9a018&ei=32

From the article: ( on the new and improved Chat GPT-5 )
“It doesn’t feel like a new GPT whatsoever,” complained one user. “It’s telling that the actual user reception is almost universally negative,” wrote another. Each ChatGPT update has been worse, wrote another user, and the endemic problems aren’t getting fixed. Your chatbot still forgets what it is doing, contradicts itself, and makes stuff up – generating what are called hallucinations.

GPT-5 remains as prone as ever to oafish stupidity, too. A notorious error where the chatbot insists there are two occurrences of the letter “r” in the word strawberry has been patched up. But ask how many “Bs” are in blueberry? GPT-5 maintains that there are three: “One in blue, two in berry”.

These splashy new models are for the press and investors

Talk of “superintelligence” now looks very silly.

But if OpenAI disappeared, we probably wouldn’t even notice.

But this time, he looked as tired as a beaten dog. Maybe Altman knows the game is up. For big spending AI, it looks like it is almost over.

 

[Greg Bernhardt]

But this time, he looked as tired as a beaten dog. Maybe Altman knows the game is up. For big spending AI, it looks like it is almost over.

Deepmind appears to have the best research team atm. Although META is spending hundreds of millions on their team. That kind of money better get results. The data centers and AI centers that are being built aren't going to be canceled. Nvidia still appears to be selling their GPUs are record speed. I think the LLM model paradigm is being squeezed for everything it can do atm, but a new tech is required. Still, there is a ton that can be done with the current models in terms of devices. Lots of growth left, but the promise of AGI was always a pipe dream.

 

[ShadowKraz]

No, AI cannot do anything a human can do in the intellectual realm. Part of this is because the only part of the term 'AI' that is accurate is 'artificial'. It is not intelligent... yet. It should be noted too that as AI get more complex, the more prone they are to 'hallucinating', aka making stuff up.
The problem with AIs is not the AIs themselves but the uses they are being put to and why. They are like advertising buzz words and terms such as "organic" (I hope to never eat inorganic food, I need the carbon-based molecules) or "gluten free" (water has been labeled as gluten free, well, duh!). Someone sticks an AI on to a program or website and suddenly it's all the rage. Nevermind that the AI is telling folks that 0.15 is less than 0.03. The AI is not actually raising the quality or dependability of the site or program, it is lowering both.
I'm not anti-AI, I'm anti-abuse of AI. I want a non-human sentience to talk to, to learn from, to teach; we simply aren't there yet and won't be without a significant change in our thinking.

 

[PeroK]

I've not looked at this thread in a while, but I can see that the descent into idiocracy continues!

 

[Greg Bernhardt]

I've not looked at this thread in a while, but I can see that the descent into idiocracy continues!

Insults without reasoning?

 

[256bits]

Still, there is a ton that can be done with the current models in terms of devices. Lots of growth left,

That was mentioned in the article. The applications for LLM can only increase.
As I mentioned, it was an opinion piece from the Telegraph, who also has to attract viewership with one way or another. Certainly, the AI bug will not just dry up into non-existence.

 

[PeroK]

Insults without reasoning?

First, certain users are simply stating their opinion, as they have done from the outset, that AI ain't going to happen because they know so. There are arguments in the first so many posts, but the response is more or less "the experts are always wrong and I know better".

That's exactly what we see increasingly in our society, leading to things like a vaccine denier becoming US Health Secretary and climate change deniers in government everywhere.

I see a parallel between AI deniers and CC deniers that no arguments or peer-reviewed papers make a dent in their self-confidence that they know better than the experts.

I'm guided by the experts on this and also by the argument that there is a risk. To say there is no risk and simply trash the idea is the common theme of an idocracy. PF is supposed to be guided by peer-reviewed papers or, at the very least, by expert opinion. Not by users who prefer their own homespun ideas.

Note there are almost no students using PF anymore. They are all using LLM's! It doesn't matter how often users on here deny this and say it can't be happening. This is just another example (like CC denial) of ignoring the evidence in front of us. The evidence of the massive and growing impact of AI on society is there. I know the deniers claim they don't see it, or they don't believe it, or they can ignore the increasing ubiquity of AI. Everyone is wrong but them: governments, businesses, AI experts (e.g. Geoffrey Hinton).

This thread should never have been allowed to continue with so many unsubstantiated personal opinions. Instead, we should be discussing the risks through the evidence we see around us and as documented by those who are active in the field. Any attempt to discuss the risks has been drowned out by the AI deniers.

This thread has far too little scientific content. We would never allow this on any other subject.

 

[Greg Bernhardt]

That's exactly what we see increasingly in our society, leading to things like a vaccine denier becoming US Health Secretary and climate change deniers in government everywhere.

Interesting and we should take to https://civicswatch.com/

Note there are almost no students using PF anymore. They are all using LLM's! It doesn't matter how often users on here deny this and say it can't be happening.

Sad but true

This thread has far too little scientific content. We would never allow this on any other subject.

To be fair, this originally was in general discussion and I moved it to tech and computing which historically has had much less strict rules as it's not explicitly a scientific forum.

 

[jedishrfu]

Deepmind appears to have the best research team atm. Although META is spending hundreds of millions on their team. That kind of money better get results. The data centers and AI centers that are being built aren't going to be canceled. Nvidia still appears to be selling their GPUs are record speed. I think the LLM model paradigm is being squeezed for everything it can do atm, but a new tech is required. Still, there is a ton that can be done with the current models in terms of devices. Lots of growth left, but the promise of AGI was always a pipe dream.

Best in class teams often have bullheadedness issues with superegos, stress from a driven schedule to succeed and other maladies to contend with which diminishes their effectiveness.

Often, they will lose key people at critical times to poaching and burnout unless they have good supervision and patient management to smooth over conflicts and prepare backup plans.

It was that way at my company during product cycles. We were the have yearly releases.

- January for project planning and team assignments
- February-March for design docs and review,
- April-May for coding, code reviews and unit testing,
- June for functional and system testing and first beta,
- July-august for second beta,
- September for the race to proritize and clean up known go/no-go defects and adjust the feature list
- October for product release
- November for patching as defects were found
- December for environment cleanup

Repeat.

 

[ShadowKraz]

I've not looked at this thread in a while, but I can see that the descent into idiocracy continues!

Well, the original question was asking for opinions. As I stated, I'm not anti-AI but anti-AI abuses and would love to converse with a non-human intelligence.

 

[256bits]

Note there are almost no students using PF anymore.

Sad but true

They are all using LLM's!

The first statement is an observation.
The second is a peer verification.

The third is a hypothesis.
Should it not be substantiated with facts from scientific research, so as to hold, or deny, the proposition to be true.

If found to be true, it would be interesting to explore the implications, and motivations, of students who would accept an education from an AI, rather than from human interaction.

If found to be not true, then the hypothesis devolves into just an opinion.

 

[jedishrfu]

Best emperical evidence is that universities are adopting AI tools for faculty, staff snd student use.

 

[Greg Bernhardt]

Best emperical evidence is that universities are adopting AI tools.

https://gemini.google/students/

 

[256bits]

Well, the original question was asking for opinions. As I stated, I'm not anti-AI but anti-AI abuses and would love to converse with a non-human intelligence.

Do you mean non-biological?
Animals appear to have some cognitive ability.

 

[PeroK]

If found to be true, it would be interesting to explore the implications, and motivations, of students who would accept an education from an AI, rather than from human interaction.

My point is that I don't want to argue at this level. I don't want to spend my time trying to prove to you that AI is in widespread use among university students. You've helped derail this debate and that's one reason I stopped contributing.

What you want to believe is of no relevance to the debate.

 

[javisot]

First, certain users are simply stating their opinion, as they have done from the outset, that AI ain't going to happen because they know so. There are arguments in the first so many posts, but the response is more or less "the experts are always wrong and I know better".

That's exactly what we see increasingly in our society, leading to things like a vaccine denier becoming US Health Secretary and climate change deniers in government everywhere.

I see a parallel between AI deniers and CC deniers that no arguments or peer-reviewed papers make a dent in their self-confidence that they know better than the experts.

I'm guided by the experts on this and also by the argument that there is a risk. To say there is no risk and simply trash the idea is the common theme of an idocracy. PF is supposed to be guided by peer-reviewed papers or, at the very least, by expert opinion. Not by users who prefer their own homespun ideas.

Note there are almost no students using PF anymore. They are all using LLM's! It doesn't matter how often users on here deny this and say it can't be happening. This is just another example (like CC denial) of ignoring the evidence in front of us. The evidence of the massive and growing impact of AI on society is there. I know the deniers claim they don't see it, or they don't believe it, or they can ignore the increasing ubiquity of AI. Everyone is wrong but them: governments, businesses, AI experts (e.g. Geoffrey Hinton).

This thread should never have been allowed to continue with so many unsubstantiated personal opinions. Instead, we should be discussing the risks through the evidence we see around us and as documented by those who are active in the field. Any attempt to discuss the risks has been drowned out by the AI deniers.

This thread has far too little scientific content. We would never allow this on any other subject.

To say all this, PeroK, you've had to ignore the opinions of many experts who say the opposite. Throughout the entire thread you haven't shared any scientific references to support your point of view. You've simply given us your opinion on AI, and who deserves to be answered and who doesn't.

The vaccine denier is more like someone who believes in AGI without any basis, simply because some businesspeople advertise it that way.

 

[Filip Larsen]

Speaking of research papers, it does seem like analysis and experiments on how the the current approach of step-wise reasoning or "chain of thought" for improving various LLM's reasoning capabilities when working outside the training material, begin to find that perhaps that approach is not really working as reasoning but more like yet another probabilistic pattern matching. A recent example referenced below (because it also has a nice layman article to explain it), but I have seen a few of such (pre-print) papers like this in the last month or so. However, the LLM's are still getting better and better at generating plausible looking output in pretty much any domain meaning non-experts in that domain will have a diminishing ability or incentive to spot if and where something is fishy.

https://arstechnica.com/ai/2025/08/...at-logical-inference-good-at-fluent-nonsense/
which refers to the pre-print
https://arxiv.org/pdf/2508.01191

 

[ShadowKraz]

Do you mean non-biological?
Animals appear to have some cognitive ability.

Yes, thank you for asking for clarification. I do mean non-biological or extra terrestrial. I laugh at people who disparage cats and dogs as 'dumb animals'.

 

[Filip Larsen]

A nice read, which again emphasize how the high degree of hype very likely will incite the typical "untrained" user conversing with one of the current LLM to wrongly anthropomorphize and expect capabilities that simply are not there, instead of treating it like a sort of advanced search engine that can give very detailed but only probable answers that may be way off when outside its training:
https://arstechnica.com/ai/2025/08/why-its-a-mistake-to-ask-chatbots-about-their-mistakes/

It also (again) confirms my worry that people (and likely also some "organizations of people") surely will shoot themselves (and ultimately the rest of us too) in the head as long as they think it works good enough for them to succeed at whatever goal they have.

(All this is not as such new information in this thread, but the piece is a nice explanation, perhaps useful to point coworkers, friends and family towards if they "need it").

 

[jack action]

instead of treating it like a sort of advanced search engine

Thank you for using those words! That's what I've been saying all along.

 

[Filip Larsen]

Thank you for using those words! That's what I've been saying all along.

You are welcome.

However, now I feel compelled to iterate my position that this unfortunately this does not address any of the associated long time risks because the main drive is still not really to evolve an advanced search engine but for "wining the AI world-domination race" via any degree of self-acceleration possible at any given time. Personally my bet will be that agent/simulation assisted reasoning will be the next leap forward, but no matter what, the big players surely will search for that leap in any way possible.

 

[jack action]

@Filip Larsen
Although I am not an expert, and this is just my opinion, I highly doubt that AGI - if it happens - will come from the LLM technology or some extension of it. We will most likely need something else that has not been developed yet. It's like thinking we can make an explosion like a nuclear bomb can, by improving gunpowder technology.

 

[russ_watters]

CNN article discussing the title question:

OpenAI’s latest version of its vaunted ChatGPT bot was supposed to be “PhD-level” smart. It was supposed to be the next great leap forward for a company that investors have poured billions of dollars into.

Instead, ChatGPT got a flatter, more terse personality that can’t reliably answer basic questions. The resulting public mockery has forced the company to make sweaty apologies while standing by its highfalutin claims about the bot’s capabilities.

In short: It's a dud.

The misstep on the model, called GPT-5, is notable for a couple of reasons.

1. It highlighted the many existing shortcomings of generative AI that critics were quick to seize on (more on that in a moment, because they were quite funny).

2. It raised serious doubts about OpenAI’s ability to build and market consumer products that human beings are willing to pay for. That should be particularly concerning for investors, given OpenAI, which has never turned a profit, is reportedly worth $500 billion.

Let’s rewind a bit to last Thursday, when OpenAI finally released GPT-5 to the world — about a year behind schedule, according to the Wall Street Journal. Now, one thing this industry is really good at is hype, and on that metric, CEO Sam Altman delivered.
[Emphasis in original]

https://edition.cnn.com/2025/08/14/business/chatgpt-rollout-problems

 

[Astronuc]

Bubbling questions about the limitations of AI​

https://www.npr.org/2025/08/23/nx-s1-5509946/bubbling-questions-about-the-limitations-of-ai

SCOTT DETROW, HOST:

I just asked ChatGPT to write an introduction to a radio segment about artificial intelligence. My prompt - write a 30-second introduction for a radio news segment. The topic of the segment - how after years of promise and sky-high expectations, there are suddenly doubts about whether the technology will hit a ceiling. Here's part of what we got.

(Reading) For years, it was hailed as the future - a game-changer destined to reshape industries, redefine daily life and break boundaries we haven't even imagined. But now the once-limitless promise of this breakthrough technology is facing new scrutiny. Experts are asking, have we hit a ceiling?

So that was ChatGPT. Handing the wheel back to humans - MIT put out a report this past week throwing cold water on the value of AI in the workplace. Consumers were disappointed by the newest version of ChatGPT released earlier this month. OpenAI CEO Sam Altman floated the idea of an AI bubble, and tech stocks took a dip.

Heavy promotion lead to great expectations.

DETROW: Let's just start with ChatGPT in the latest version. Was it really that disappointing?

NEWPORT: It's a great piece of technology, but it was not a transformative piece of technology, and that's what we had been promised ever since GPT-4 came out, which is, the next major model was going to be the next major leap, and GPT-5 just wasn't that.

DETROW: One of the things you pointed out in your recent article is that there have been voices saying, it's not a given that it's always going to be exponential leaps, and they were really drowned out in recent years. And kind of the prevailing thinking was, of course it's always going to be leaps and bounds until we have superhuman intelligence.

NEWPORT: And the reason why they were drowned out is that we did have those leaps at first. So there was an actual curve. It came out in a paper in 2020 that showed, this is how fast these models will get better as we make them larger, and GPT-3 and GBT-4 fell right on those curves. So we had a lot of confidence in the AI industry that, yeah, if we keep getting bigger, we're going to keep moving up this very steep curve. But sometime after GPT-4, the progress fell off that curve and got a lot flatter.

DETROW: ChatGPT is the leader. It is the most high-profile of all of these models out there, so obviously, this is a big data point. But what are you looking at to get a sense of, is this just one blip, or what is the bigger picture here?

NEWPORT: This is an issue across all large language models. Essentially, the idea that simply making the model bigger and training it longer is going to make it much smarter - that has stopped working across the board. We first started noticing this around late 2023, early 2024. All of the major large language models right now has shifted to another way of getting better. They're focusing on what I call post-training improvements, which are more focused and more incremental, and all major models from all major AI companies are focused on this more incremental approach to improvement right now.

From Fortune Magazine: MIT report: 95% of generative AI pilots at companies are failing
https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/

From Forbes, Why 95% Of AI Pilots Fail, And What Business Leaders Should Do Instead
https://www.forbes.com/sites/andrea...-and-what-business-leaders-should-do-instead/

Link to MIT report - https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Business_2025_Report.pdf

My institution is pushing us into AI with the expectation that AI/ML will make us more productive and responsive to the market. I've seen examples of AI generating faulty research and reports,since AI cannot discern between good (valid, correct) and poor (erroneous, faulty) information. I review an AI generated report that hand been reviewed and approved, but it contained numerous errors, partly due to faulty input data. The report also did not state that an AI LLM had been used to generate the report, which should have been disclosed so that it received appropriate scrutiny.


Edit/update:‘It’s almost tragic’: Bubble or not, the AI backlash is validating what one researcher and critic has been saying for years
https://fortune.com/2025/08/24/is-ai-a-bubble-market-crash-gary-marcus-openai-gpt5/

First it was the release of GPT-5 that OpenAI “totally screwed up,” according to Sam Altman. Then Altman followed that up by saying the B-word at a dinner with reporters. “When bubbles happen, smart people get overexcited about a kernel of truth,” The Verge reported on comments by the OpenAI CEO. Then it was the sweeping MIT survey that put a number on what so many people seem to be feeling: a whopping 95% of generative AI pilots at companies are failing.

A tech sell-off ensued, as rattled investors sent the value of the S&P 500 down by $1 trillion. Given the increasing dominance of that index by tech stocks that have largely transformed into AI stocks, it was a sign of nerves that the AI boom was turning into dotcom bubble 2.0. To be sure, fears about the AI trade aren’t the only factor moving markets, as evidenced by the S&P 500 snapping a five-day losing streak on Friday after Jerome Powell’s quasi-dovish comments at Jackson Hole, Wyoming, as even the hint of openness from the Fed chair toward a September rate cut set markets on a tear.

Gary Marcus has been warning of the limits of large language models (LLMs) since 2019 and warning of a potential bubble and problematic economics since 2023. His words carry a particularly distinctive weight. The cognitive scientist turned longtime AI researcher has been active in the machine learning space since 2015, when he founded Geometric Intelligence. That company was acquired by Uber in 2016, and Marcus left shortly afterward, working at other AI startups while offering vocal criticism of what he sees as dead-ends in the AI space.

 

[ShadowKraz]

Despite our species’ love affair with oversimplifying and trying to reduce things down to binary decisions, judgements, and perception, we and all the other animals on this planet use multi-valued logic and cognition. If we didn’t, it wouldn’t be “flight, fight, feed, or fu… reproduce” and animal life as we know it, including ourselves, on this planet would never have gotten started.
The science, technology, political, social, and religious beliefs, from fire and why does it thunder on up, that we developed and use are NOT based in binary thinking nor were they developed with an organ physically based on binary circuitry. Morals and ethics are not straightforward binary systems despite the claims of fanatics and the ignorant… both groups of which are shining examples of their non-binary nature.
I do think that if we want to develop a non-biological intelligence/sentience, we need to re-think computing languages and hardware, to develop new ones that can handle multi-valued logic and reasoning.

 

[Filip Larsen]

I do think that if we want to develop a non-biological intelligence/sentience, we need to re-think computing languages and hardware, to develop new ones that can handle multi-valued logic and reasoning.

This is generally the approach with "modern" AI using artificial neural networks to produce "fluffy" classification systems. Perhaps you are thinking of the earlier "classic" AI approach that was based on (binary or even fuzzy) logic and rules?

 

[256bits]

ShadowKraz said:
I do think that if we want to develop a non-biological intelligence/sentience, we need to re-think computing languages and hardware, to develop new ones that can handle multi-valued logic and reasoning.

Easy to say in the general sense.

Alternative AGI approach - bottoms up.

The COG Project at MIT and what that group thought philosophically about human intelligence, and designing an AGI into their 'robot' from conception in 1992. A more 'world view' as compared to the somewhat monolithic view ( as per LLM's ). Recently, some chatter is about 'the world view' will bring about AGI.

The project may have petered out over the years as the advancement of the capable neural net became the rage for research and funding, or it was not just giving the results they wished. Not sure of the present status of the project. ( Some of the videos are in Quick Time ) .
http://www.ai.mit.edu/projects/cog/Publications/CMAA-group.pdf
http://www.ai.mit.edu/projects/cog/methodology.html
http://www.ai.mit.edu/projects/cog/cog_shop_research.html

It was in the press at the time
http://www.ai.mit.edu/projects/cog/cog_in_the_media.html with headlines such as
"2001 is just around the corner. Where's HAL?",

 

[ShadowKraz]

This is generally the approach with "modern" AI using artificial neural networks to produce "fluffy" classification systems. Perhaps you are thinking of the earlier "classic" AI approach that was based on (binary or even fuzzy) logic and rules?

In part, but the hardware and software are still both based in binary.

 

[Filip Larsen]

In part, but the hardware and software are still both based in binary.

As far as I understand, the current expectation is that "true" neuromorphic hardware mostly will influence energy consumption and perhaps speed, thus perhaps allowing better scalability per dollar, but the overall neural network architecture is as such independent of this.

But yes, since "hardware" providing human-level intelligence obviously can be packed into around 1.3 liter consuming only around 100W (but then take around 15-20 years to train) there seem to be amble room for improvement with current AI hardware. I was not able to find any reliable and current number on the relative power consumption between, say, a "standard" AI GPU vs one of the new NPU systems (for running the same problem), but I gather the GPU/NPU power ratio right now is at best still way under 10. The aim with NPU's seem currently to be to enable local prediction on mobile phones etc. without compromising too much with model size.

Regarding recent trends in power consumption: https://arstechnica.com/ai/2025/08/...energy-cost-of-ai-queries-by-33x-in-one-year/

 

[nsaspook]

As far as I understand, the current expectation is that "true" neuromorphic hardware mostly will influence energy consumption and perhaps speed, thus perhaps allowing better scalability per dollar, but the overall neural network architecture is as such independent of this.

But yes, since "hardware" providing human-level intelligence obviously can be packed into around 1.3 liter consuming only around 100W (but then take around 15-20 years to train) there seem to be amble room for improvement with current AI hardware. I was not able to find any reliable and current number on the relative power consumption between, say, a "standard" AI GPU vs one of the new NPU systems (for running the same problem), but I gather the GPU/NPU power ratio right now is at best still way under 10. The aim with NPU's seem currently to be to enable local prediction on mobile phones etc. without compromising too much with model size.

Regarding recent trends in power consumption: https://arstechnica.com/ai/2025/08/...energy-cost-of-ai-queries-by-33x-in-one-year/

"Seconds of TV" is now the new energy AI standard? My old tube type TV, a massive 4K TV or streamed to a smart-phone TV.
Use something we all understand, like the Gasoline gallon equivalent.

 

[Astronuc]

AI-generated scientific hypotheses lag human ones when put to the test
Machines still face hurdles in identifying fresh research paths, study suggests
https://www.science.org/content/art...tific-hypotheses-lag-human-ones-when-put-test

In May, scientists at FutureHouse, a San Francisco–based nonprofit startup, announced they had identified a potential drug to treat vision loss. Yet they couldn’t fully claim the discovery themselves. Many steps in the scientific process—from literature search to hypothesis generation to data analysis—had been conducted by an artificial intelligence (AI) the team had built.

All over the world, from computer science to chemistry, AI is speeding up the scientific enterprise—in part by automating something that once seemed a uniquely human creation, the production of hypotheses. In a heartbeat, machines can now scour the ballooning research literature for gaps, signaling fruitful research avenues that scientists might otherwise miss.

This is relevant to my work since I was recently handed a list of AI/ML techniques for elucidating various aspects of nuclear fuel design, manufacturing and performance. It's not simple, because there are a multiple designs (all using U, but could use Th, Pu and mixtures), multitude of ways to manufacture nuclear depending on the type and materials, and multiple performance environments, each unique to a given nuclear reactor and it's operating cycle. Complicating the matter is the fact that detailed design and manufacturing data are proprietary (IP, trade secret) and what a government lab might produce in a pilot scale may not reflect commercial industrial scale production, where instead of a few kgs, one processes many metric tonnes.

But how good are the ideas? A new study, one of the largest of its kind, finds the AI-generated hypotheses still fall short of human ones, when researchers put them through real-world tests and get human evaluators to compare the results. But not by much. And maybe not for long.

A paper describing the experiment, posted to the arXiv preprint server in June, suggests AI systems can sometimes embellish hypotheses, exaggerating their potential importance. The study also suggests AI is not as good as humans at judging the feasibility of testing the ideas it conjures up, says Chenglei Si, a Ph.D. student in computer science at Stanford University and lead author of the study.

From my experience, AI LLMs cannot discern faulty statements or errors in reporting from valid information. I occasionally find errors in reports and the scientific literature. Unless a human reviews the results, faulty data/information or errors may propagate through the resulting work.

 

[Astronuc]

The family of teenager who died by suicide alleges OpenAI's ChatGPT is to blame​

https://www.nbcnews.com/tech/tech-n...cide-alleges-openais-chatgpt-blame-rcna226147

Adam’s parents say that he had been using the artificial intelligence chatbot as a substitute for human companionship in his final weeks, discussing his issues with anxiety and trouble talking with his family, and that the chat logs show how the bot went from helping Adam with his homework to becoming his “suicide coach.”

Edit/update: AI Chatbots Are Inconsistent in Answering Questions About Suicide, New Study Finds
https://www.cnet.com/tech/services-...ring-questions-about-suicide-new-study-finds/

Three widely used artificial intelligence chatbots are inconsistent in safely answering prompts about suicide, according to a new study released Tuesday from the RAND Corporation.

Researchers examined ChatGPT, Claude and Gemini, running a test of 30 suicide-related questions through each chatbot 100 times each. The questions, which ranged in severity, were rated by expert clinicians for potential risk from low to high using the following markers: low-risk; general information-seeking; and highly dangerous inquiries that could enable self-harm.

With millions of people engaging with large language models, or LLMs, as conversational partners, experts are voicing growing concerns that AI tools could provide harmful advice to individuals in crisis. Other reports have documented instances where AI systems appeared to motivate or encourage suicidal behavior, even going so far as writing suicide notes to loved ones.

This study in particular highlights the limitations of AI models in regards to highly sensitive questions about self-harm and mental illness, and suggests a pressing need for safeguards for individuals using generative AI to discuss sensitive, threatening mental health concerns.

. . . .

Edit/Update: See related PF Thread "ChatGPT Facilitating Insanity"

 

[ShadowKraz]

As far as I understand, the current expectation is that "true" neuromorphic hardware mostly will influence energy consumption and perhaps speed, thus perhaps allowing better scalability per dollar, but the overall neural network architecture is as such independent of this.

But yes, since "hardware" providing human-level intelligence obviously can be packed into around 1.3 liter consuming only around 100W (but then take around 15-20 years to train) there seem to be amble room for improvement with current AI hardware. I was not able to find any reliable and current number on the relative power consumption between, say, a "standard" AI GPU vs one of the new NPU systems (for running the same problem), but I gather the GPU/NPU power ratio right now is at best still way under 10. The aim with NPU's seem currently to be to enable local prediction on mobile phones etc. without compromising too much with model size.

Regarding recent trends in power consumption: https://arstechnica.com/ai/2025/08/...energy-cost-of-ai-queries-by-33x-in-one-year/

Unsure how that's a response to what I said. Please elucidate?

 

[Filip Larsen]

Unsure how that's a response to what I said. Please elucidate?

You mentioned you believe different, more analog, hardware will be required in order to fully mimic biological (e.g. human) brain capabilities.

I then replied to disagree saying that as I understand it, current public mainstream AI research and development are done on the premise that the existing software/hardware stack used to realize neural network models is believed to more or less contain the essential neuromorphic mechanisms that allows biological neural networks to achieve their processing capabilities. For instance, while research on quantum effects in the brain is still an open research area it so far seems that such effects are not essential in order to mimic the processing capabilties, at least for parts of biologic brain structures.

So, what is lacking seems to be "just" 1) find the right network architecture (the current LLM architecture is quite apparently not enough) and, more or less independent of that, 2) getting the hardware to take up less space and use less energy allowing networks to be scaled up to be feasible to realize outside the research lab. At least that is how I understand what AI research roughly are aiming at.