Is AI Overhyped?

[russ_watters]

The question is, how might the bursting of the AI/AGI bubble affect the global economy, given its widespread integration into society?

What widespread integration into society? I haven't seen that.

My expectation is that the bursting of the AI bubble will be similar to the bursting of the tech bubble in 2000. It will be a major stock market crash for indices heavily invested in it, but will have very little impact beyond that (the 2001 recession barely registered as a recession). Unprofitable companies will go out of business and companies that are financially sound will survive and continue.

 

[Astronuc]

After Laying Off 8,000 Employees, Zuckerberg Admits Meta’s AI ‘Hasn’t Really Accelerated’ As Expected
https://www.yahoo.com/finance/technology/ai/articles/laying-off-8-000-employees-121545621.html

At an internal Meta town hall on July 2, 2026, CEO Mark Zuckerberg told employees that AI agent development over the prior four months "hasn't really accelerated in the way that we expected," per a recording heard by Reuters. He added that the company's reorganization was not as "clean" as planned and that its bets on the new structure "haven't come to fruition yet," though he expects meaningful benefits within three to six months.

Seems like a case of unrealistic expectations.

 

[gmax137]

I wouldn't be happy if the company I worked for laid off 8000 employees. To me, that shows a lack of serious planning. But, I worked for the same outfit for over 40 years. So I'm probably the one with unrealistic expectations.


Wiki says Meta has 77,986 employees (March 2026).

 

[Hornbein]

Post 87 billion dollars Meta pulled the plug on the virtual Metaverse. They couldn't compete with Genshin Impact. The staff for the project are superfluous.

 

[russ_watters]

Post 87 billion dollars Meta pulled the plug on the virtual Metaverse. They couldn't compete with Genshin Impact. The staff for the project are superfluous.

Yeah, I guess they were hoping to re-task them to AI and it didn't pan out.

 

[Astronuc]

The AI Productivity Argument Is Over​

You can 10x or 100x or 1000x all you want, but that’s not the productivity boost you’re looking for.
https://www.inc.com/joe-procopio/the-ai-productivity-argument-is-over/91372216

This whole myth started to unravel over a year ago, when reports started surfacing that once the actual measurement of AI productivity had been made, the results were—well, let’s say far from spectacular.

How about that oft-quoted MIT study?

“Just 5 percent of integrated AI pilots are extracting millions in value, while the vast majority remain stuck with no measurable P&L impact.”

It's an interesting read. AI cannot generate original ideas from scratch, it mimics/copies/regurgitates.

those critics of the AI critics weren’t necessarily wrong. A good percentage of the problems that the enterprise is having with their AI implementations is not AI’s fault. The truth, the quiet part I’m about to say out loud: We’re simply not ready for that much productivity.

 

[Greg Bernhardt]

My expectation is that the bursting of the AI bubble will be similar to the bursting of the tech bubble in 2000. It will be a major stock market crash for indices heavily invested in it, but will have very little impact beyond that (the 2001 recession barely registered as a recession). Unprofitable companies will go out of business and companies that are financially sound will survive and continue.

I think it will be sparked by open source models. GLM-5.2 is good enough for most light to medium scale tasks. Once there are frontier open source models, companies will understand they don't need to spend hundreds of millions on closed models.

https://www.vellum.ai/open-llm-leaderboard

 

[Jimmy87]

The problem with the bubble isn't just that AI has been oversold (which too); it's that they're essentially selling the concept of AGI before it even exists. The bigger the bubble, the worse the consequences will be if it bursts. The question is, how might the bursting of the AI/AGI bubble affect the global economy, given its widespread integration into society?

Do you not think there is a fairly large consensus among the top experts in the field that AGI/ASI will likely arrive within the next few years? I agree the bubble will burst at some point but I can’t see the pursuit of AGI/ASI stopping.

 

[javisot]

Do you not think there is a fairly large consensus among the top experts in the field that AGI/ASI will likely arrive within the next few years?

Which "top expert" are you referring to, and who do they work for? If they don't work for major AI companies, what are they trying to achieve by proclaiming such outlandish things? Attention? Perhaps a job? Or do they simply enjoy scaring people with Skynet-like scenarios?

It is very likely that we will see the definition of AGI degenerate as much as necessary in order to say that we have achieved it.

 

[Borg]

It is very likely that we will see the definition of AGI degenerate as much as necessary in order to say that we have achieved it.

I think that the opposite is actually happening. The Turing Test was originally a test to determine whether a machine could conduct a text conversation well enough that a human judge could not reliably distinguish it from another human. There have been documented tests showing that modern LLMs have already passed the original definition - Large Language Models Pass the Turing Test. Even before that paper though, the demanded evidence has shifted to other tests:

• genuine “understanding,” rather than just believeable conversations
• autonomous goals and long-term planning
• long-term memory and the "ability to learn" which is often vaguely defined.
• competence across a wider range of capabilities than most humans.
• true consciousness - again, vaguely defined.

I see AGI as no different from how the original Turing Test was treated. People set the original goal, the models get close, and the goalposts move.

 

[javisot]

I see AGI as no different from how the original Turing Test was treated. People set the original goal, the models get close, and the goalposts move.

Good example. The Turing test is another case of a degenerate concept riddled with ambiguity. For example, ChatGPT passes the Turing test, but I can very accurately recognize when I'm communicating with ChatGPT (you probably also detect ChatGPT very effectively, and paradoxically, it's a model that passes the Turing test).

How many human beings do we know who communicate solely in a one-to-one relationship, one input to one output?

Human communication is far more complex and chaotic.

 

[Borg]

How many human beings do we know who communicate solely in a one-to-one relationship, one input to one output?

How else would you do the experiment?

 

[Jimmy87]

Which "top expert" are you referring to

Most of the world's top AI alignment scientists. Most of them are serious and devoid of hype but I would pick Geoffrey Irving as the most credible since he is not an AI alignment scientist for a frontier lab. He is a civil servant for the UK government.

who do they work for?

He works for the UK government's AI Security Instutite (AISI). The largest independent, government-run AI safety body in the world.

If they don't work for major AI companies, what are they trying to achieve by proclaiming such outlandish things? Attention? Perhaps a job? Or do they simply enjoy scaring people with Skynet-like scenarios?

He is very measured and conservative so I wouldn't say any of this describes him nor his remit as the lead AI safety scientist working as a civil servant for the AISI.

He was interviewed here in June:

Go to 33 minutes where he says recursive self improvement and superintelligence is 2-3 years away. He does say there is uncertainty with that and he hopes he is wrong.

He has just left AISI citing that AI has progressed much faster and he thinks he can now make the biggest difference going back to theory work since aligning AGI/superintelligence is currently largely unsolved and he is very skeptical of automated alignment working.

Going back to your hype and scare point that a lot of tech CEOs have - Irving, in this interview, refuses to even give a p doom -



9 mins 30s to around 20 mins outlines the current frontier risk of current AI systems and how good the current mitigations are. At 19 mins 30s he refuses to give a p doom.

I haven't visited this forum for a long while but having looked at the AI threads around this sort of topic I encourage people to read the scientific reports that regularly come out of the AISI. In a field ruled by immoral tech titans, a government run AI safety body that reguarly outputs scientific reports is crucial. METR also do this but AISI is more broader. AISI also has some of the world's best AI scientists working there. Xander Davies is a world class AI red team scientist (despite his incredibly young age). He jailbroke all the frontier LLMs despite the red teams at those companies saying they couldn't do it. By jailbreak I don't mean it is forced to misbehave; I mean full jailbreaking where he got one to give them a homemade recipe for Anthrax (https://global.foresightnews.pro/article/16060).

Xander Davies on jailbreaking frontier LLMs -

Here is a link to the AISI (https://www.aisi.gov.uk/blog).

Here is there most recent summary of how capable they think Mythos is - https://www.aisi.gov.uk/blog/our-evaluation-of-claude-mythos-previews-cyber-capabilities

I've also attached their May report on the limitations of current monitoring and safety mitigations for frontier AI. This is the reason Irving last left to go find new breakthroughs in alignment.

 

[Filip Larsen]

Human communication is far more complex and chaotic.

What does that have to do with measurable progress towards AGI or even ASI, i.e. systems that intelligently can solve problems on-par or even superseeding the average ("IQ 100") human?

A large part of the discussion about AI on PF seems to be hung up on the detail that one run of an LLM in isolation "only" mimics apparent intelligent (i.e. that the output only reads like a plausible informed answer) but miss the bigger picture of how intelligent solutions of "average" problems come about. In a nutshell, the human thought process can to some extend be said to follow the same way as (some) AI agent system. Humans train (for a long time) to improve and do tasks that requires or benefits from the application of intelligence ("intuition" and "pattern association") and when we apply those trained skills we internally generate ideas much the same as an LLM, e.g. a plausible part solution that works us closer to the problem we are trying to solve, then we apply some self-critique and possibly apply some corrections before looping on. Some times those part ideas are off, wrong or even terrible. You wouldn't say that a briliant human is non-intelligent just because he or she occasionally makes a brainfart in this process before correcting it and moving on to solve a highly complex problem. Trial and error is a cornerstone of human "every day" intelligence so why should an AI system doing exactly the same process acheiving the same or even better result not be considered an intelligent agent?

As I see it (and I feel I am certainly not alone in this view) the only hinderance of progress towards AGI currently seems to be how many (computing) resources someone are prepared to throw at it, and thus progress is not (yet, at least) limited by any fundmental "magic ingredient of intelligence" that computing technology in principle would be unable to discover or apply.

 

[gleem]

Those who dismiss AI as just a stochastic parrot fail to see that LLMs behave very much as we do, or won't admit it.

Do they understand? If they do what we ask, then they understand. Most will not say AI is conscious, but is consciousness necessary? If I'm told to go get a beer. I do not consciously think about it when I am simultaneously watching a football match. I just get it.

Do humans hallucinate? All the time, if you mean they do give incorrect or fabricated responses. Humans devise all sorts of anomalous behaviors, like inventing conspiracy theories and believing that the stars influence their lives.

AI models do well on human IQ exams. They are better than most humans at cognitive tasks.

AI models are hindered because they have no physical contact with the world; putting frontier models in robots should have some interesting consequences. There is some movement to let AI models learn like a child. So it will be interesting to see how those models develop.

Recently, Anthropic discovered a part of Claude's neural net called the J-space that seems to be a nexus of AI's processing activity, similar to what is called the Global Workspace in the human brain, which leads to our conscious thoughts. This J Space is an emergent structure

Anthropic has a short video explaining the J Space.

As AI develops, our stochastic parrot seems to be doing more than mimicking human speech.

 

[Jimmy87]

AI models do well on human IQ exams.

Would you not say more like they considerably outperform humans in terms of the latest models? AISI published a report showing they are about 1.5 times the phd baseline in the sciences and maths. They also ace the maths olympiad. Models are scoring 60% in Humanities Last Exam which is supposedly made up of questions they haven't seen in their training data. Something like a typical phd student would score less in their own field let alone across many domains.

I agree the stochastic parrot is not valid anymore. It also trivialises the models when they are getting worryingly capable. This benchmark below from AISI is a cyber security capability task. An expert human would take 20 hours to complete all 32 steps. Claude Sonnet 3.7 could only do 6 steps in 2025. Now we have models completing all 32 steps autonomously (and with less tokens!).

This week in the UK we have had warnings from the government telling us to stock up a short supply of water and food due to "Advanced AI cyber attack threats". Every major news outlet reported on this. They have updated their National Risk Register (NNR) and added three AI cyber threats to it (two are in the second highest category for likelihood - 5-25% chance over the next 2 years). Only a few months ago people were saying Mythos was overhyped PR spin by the tech companies. Both METR and AISI have said cyber capabilities of frontier LLMs are currently doubling in capability every 5 months.

Recently, Anthropic discovered a part of Claude's neural net called the J-space that seems to be a nexus of AI's processing activity, similar to what is called the Global Workspace in the human brain, which leads to our conscious thoughts. This J Space is an emergent structure

Another reason to not persue ASI in addition to the fact alignment of such a system is currently unsovled. It seems there's a non-trivial chance advanced AI (AGI/ASI) would have morally releveant experiences (not conciousness per say) yet AI companies are ok having a business model which will have them as obidient workers under human control?

 

[Borg]

Recently, Anthropic discovered a part of Claude's neural net called the J-space that seems to be a nexus of AI's processing activity, similar to what is called the Global Workspace in the human brain, which leads to our conscious thoughts. This J Space is an emergent structure

I read a bit about the J Space this week. It is an interesting discovery.

 

[Jimmy87]

Out of interest, how would people rank the difficulty of this physics exemplar from Humanities Last Exam:

 

[proteo]

The AI Productivity Argument Is Over​

You can 10x or 100x or 1000x all you want, but that’s not the productivity boost you’re looking for.
https://www.inc.com/joe-procopio/the-ai-productivity-argument-is-over/91372216



It's an interesting read. AI cannot generate original ideas from scratch, it mimics/copies/regurgitates.

Newton's metaphor of "standing on the shoulders of giants," and Einstein's work building on Lorentz and Poincaré, both point to the same thing: the history of science advances not through ideas that come from nowhere, but through the rearrangement of existing material. The human mind doesn't create out of nothing either. It combines existing concepts in new ways.


Sakana AI's "AI Scientist" system does exactly this. It scans the literature, recombines existing ideas into new hypotheses, designs experiments, and interprets results. The resulting paper passed peer review at a top-tier ML conference, meaning the output was judged by human peers to be scientifically sound.


If the standard for creativity is a pure, original idea that comes from nowhere, neither the human scientist nor the AI Scientist meets it. Both rearrange what already exists. If there's a difference, it isn't in the process itself. It may lie only in whether there's a purpose or intuition guiding which combination counts as good.


In Liu Cixin's The Three-Body Problem, sophons traumatize human scientists by locking down fundamental physics research, rendering them unable to progress. The fear underneath that scene isn't really about sophons. It's about scientists discovering they're no longer necessary. AI raises the same fear today, but from a different direction: not by blocking human discovery, but by potentially doing it instead.