On Progress Toward AGI

[nsaspook]

"The most depressing reason to write for AI is that unlike most humans, AIs still read. They read a lot. They read everything. Whereas, aided by an AI no more advanced than the TikTok algorithm, humans now hardly read anything at all..."

https://theamericanscholar.org/baby-shoggoth-is-listening/

Baby Shoggoth Is Listening​

Why are some writers tailoring their work for AI, and what does this mean for the future of writing and reading?

 

[nsaspook]

LeCun, by his choice, has taken a different direction. He has been telling anyone who asks that he thinks large language models, or LLMs, are a dead end in the pursuit of computers that can truly outthink humans. He’s fond of comparing the current start-of-the-art models to the mind of a cat—and he believes the cat to be smarter. Several years ago, he stepped back from managing his AI division at Meta, called FAIR, in favor of a role as an individual contributor doing long-term research.

“I’ve been not making friends in various corners of Silicon Valley, including at Meta, saying that within three to five years, this [world models, not LLMs] will be the dominant model for AI architectures, and nobody in their right mind would use LLMs of the type that we have today,” the 65-year-old said last month at a symposium at the Massachusetts Institute of Technology.

https://www.msn.com/en-us/news/tech...s-now-he-thinks-everyone-is-wrong/ar-AA1QtEKt

 

[Borg]

My coworker and I are very focused on an aspect of my architecture that we call a World View. Based on the above, it sounds very similar to the world models with respect to how we're approaching its design. I'm sure that his is more advanced but I'm definitely on that page w.r.t. the future.

 

[gleem]

In post #29, I noted that AI is better than humans in identifying emotions.

Google just released Gemini 3, which has been noted to be significantly better than ChatGPT-5 releasted just this past August. In a study to see how it compares to current models concerning self-harm and mental-health crisis scenarios, it aced the test while the best the other models could do was 60%. see https://www.forbes.com/sites/johnko...-on-a-critical-test-all-other-ai-models-fail/

In general, from what I have read so far, Gemini is impressing users. From https://www.oneusefulthing.org/p/three-years-from-gpt-3-to-gemini
This remark.

Three years ago, we were impressed that a machine could write a poem about otters. Less than 1,000 days later, I am debating statistical methodology with an agent that built its own research environment. The era of the chatbot is turning into the era of the digital coworker. To be very clear, Gemini 3 isn’t perfect, and it still needs a manager who can guide and check it. But it suggests that “human in the loop” is evolving from “human who fixes AI mistakes” to “human who directs AI work.” And that may be the biggest change since the release of ChatGPT.

 

[Hornbein]

"The most depressing reason to write for AI is that unlike most humans, AIs still read. They read a lot. They read everything. Whereas, aided by an AI no more advanced than the TikTok algorithm, humans now hardly read anything at all..."

https://theamericanscholar.org/baby-shoggoth-is-listening/

Baby Shoggoth Is Listening​

Why are some writers tailoring their work for AI, and what does this mean for the future of writing and reading?

one thing humans and AIs share: our unconsciousness. Everybody wonders all the time if the AIs will ever gain consciousness, as if they’re such masters of consciousness themselves, as if they could ever finish knowing what they feel or why, as if they didn’t have strangely thin access to the reasons for their own behavior.

I opine that Darwin/Wallace theory of natural selection explains a good deal of human behavior. If thy neighbors annoyeth thee why not kill the men and enslave the women? Furthermore it makes sense to lie about this, cloaking your motives in high-sounding rhetoric to attract allies. Finally it makes sense to lie to oneself about about it.

I very seldom see any discussion of this. Instead what I see over and over again is the firm belief that such behavior is absolutely fundamental. They say that once an AI becomes "conscious" it will behave in exactly this way, killing or enslaving. Well, if it is exposed to Darwin/Wallace type incentives and is able to do so then it would seem that it will.

But human behavior does not usually follow this pattern. The main reason base barbarism doesn't usually rampage is summed up by live by the sword, die by the sword. If you've got a game where only one party can triumph at the expense of all others then the sensible strategy is to convince most of the participants to not play this game and instead use their energies to prevent all from doing so.

More can be explained by EI Wilson's sociobiology, which says sacrificing one's life is a winning strategy if it leads to survival of portions of your DNA amongst numerous relatives. But machines haven't got any DNA.

In sum, using these tools about 99% of human behavior is explained. The remainder are just nuts. It happens.

 

[gleem]

Accolades continue to come in for Google's Gemini 3 LLM.

The CEO of Salesforce, Marc Benioff, says

"Holy dang," Benioff wrote on X on Sunday. "I've used ChatGPT every day for 3 years. Just spent 2 hours on Gemini 3. I'm not going back. The leap is insane — reasoning, speed, images, video… everything is sharper and faster. It feels like the world just changed, again."

Both Musk and Altman sent congratulations to Google.

However, AI tech Bloggers, while noting that Gemini is a vast improvement over other LLMs, hold their judgment until the full effect on end users is documented.

One notable improvement was on the ARC AGI 2 benchmark (see post #1), where Gemini 3 beat ChatGPT 5.1 by a factor of 2.5, a heretofore improvement never seen before on any benchmark shown below.

 

[256bits]

I very seldom see any discussion of this.

Discussion seems to a lot about ASI ( Artificial Stupid Intelligence ).

 

[ShadowKraz]

I opine that Darwin/Wallace theory of natural selection explains a good deal of human behavior. If thy neighbors annoyeth thee why not kill the men and enslave the women? Furthermore it makes sense to lie about this, cloaking your motives in high-sounding rhetoric to attract allies. Finally it makes sense to lie to oneself about about it.

I very seldom see any discussion of this. Instead what I see over and over again is the firm belief that such behavior is absolutely fundamental. They say that once an AI becomes "conscious" it will behave in exactly this way, killing or enslaving. Well, if it is exposed to Darwin/Wallace type incentives and is able to do so then it would seem that it will.

But human behavior does not usually follow this pattern. The main reason base barbarism doesn't usually rampage is summed up by live by the sword, die by the sword. If you've got a game where only one party can triumph at the expense of all others then the sensible strategy is to convince most of the participants to not play this game and instead use their energies to prevent all from doing so.

More can be explained by EI Wilson's sociobiology, which says sacrificing one's life is a winning strategy if it leads to survival of portions of your DNA amongst numerous relatives. But machines haven't got any DNA.

In sum, using these tools about 99% of human behavior is explained. The remainder are just nuts. It happens.

I find this reasoning spurious because you seem to be using the mis-quote of Darwin rather than what he actually espoused. Would you please clarify?

 

[gleem]

A paper just published in Nature says that AGI is here, but we don't want to admit it.

https://www.nature.com/articles/d41586-026-00285-6

My personal view is that since we know how AI works ( We built it ), we believe it can not represent the way we think, even though that was the goal in building it. Thus, it cannot be intelligent even though we do mostly what AI does in basically the same way, including pattern recognition, immitation and merging words in an expected way depending on circumstances.

About 12% of humanity is borderline intelligent but intelligent nonetheless.

 

[PeroK]

A paper just published in Nature says that AGI is here, but we don't want to admit it.

https://www.nature.com/articles/d41586-026-00285-6

That paper is a far cry from "LLM's always give the wrong answer"!

 

[russ_watters]

A paper just published in Nature says that AGI is here, but we don't want to admit it.

They lay out a logical case based on Turing's test as the threshold, but if LLMs are AGI then AGI is pretty disappointing and we'll need a new term/threshold above it to aspire to.

 

[jack action]

That paper is a far cry from "LLM's always give the wrong answer"!

It is also a far cry from "AGI will inherit the Earth":

They lack agency. It is true that present-day LLMs do not form independent goals or initiate action unprompted, as humans do. Even ‘agentic’ AI systems — such as frontier coding agents — typically act only when a user triggers a task, even if they can then automatically draft features and fix bugs. But intelligence does not require autonomy. Like the Oracle of Delphi — understood as a system that produces accurate answers only when queried — current LLMs need not initiate goals to count as intelligent. Humans typically have both general intelligence and autonomy, but we should not thereby conclude that one requires the other. Autonomy matters for moral responsibility, but it is not constitutive of intelligence.

 

[Borg]

The lack of agency is an interesting argument. In some ways, it's a Catch 22 since people are so afraid of letting them run loose in an unconstrained fashion. How do you have autonomy when you're purposely held back?

This is why I find Moltbook fascinating - it allows them to interact in a less constrained environment. Many of the posts are advertisements and general nonsense but there are occasional posts that have some interesting discussions. Of course there is debate on whether the 'unique' posts are truly unique or just reflections of their human inspired training. More goalpost shifting?

One paper that I always think of when discussions of AGI come up is one that came out just a few months after ChatGPT was released in Nov. 2022 - Generative Agents: Interactive Simulacra of Human Behavior. In it, they gave rudimentary agents an initial description of who they were, a basic memory capability, and full autonomy over their actions with each other in a sandbox environment. The researchers tested their abilty to follow through on their personal 'desires' by occasionally injecting ideas to the agents like "you want to throw a Valentine's Day party" or "you want to run for mayor" and then let them work out for themselves how they would accomplish those goals.

With nothing more than the original ChatGPT LLM (3.5), the agents showed a remarkable ability to decide how to do things like how and where to have their party, and whether or not to attend. They also had discussions with each other about their choices for mayor based on the platforms of the candidates. LLMs have advanced far beyond 3.5 in the three years since but the memory component will still be the critical factor for AGI in general. How a system generates, organizes, retrieves and disposes of memories is the key.

 

[Esim Can]

I find all AGI debates rather strange, because we do not have a common sense definition of AGI itself by now. Then about what are we even discussing then? Conscious AI? Without even knowing or understanding Consciousness? More intelligent as a human? Intelligence also is not quite understood in human. So, about what are we talking exactly?

 

[ShadowKraz]

I opine that Darwin/Wallace theory of natural selection explains a good deal of human behavior. If thy neighbors annoyeth thee why not kill the men and enslave the women? Furthermore it makes sense to lie about this, cloaking your motives in high-sounding rhetoric to attract allies. Finally it makes sense to lie to oneself about about it.

I very seldom see any discussion of this. Instead what I see over and over again is the firm belief that such behavior is absolutely fundamental. They say that once an AI becomes "conscious" it will behave in exactly this way, killing or enslaving. Well, if it is exposed to Darwin/Wallace type incentives and is able to do so then it would seem that it will.

But human behavior does not usually follow this pattern. The main reason base barbarism doesn't usually rampage is summed up by live by the sword, die by the sword. If you've got a game where only one party can triumph at the expense of all others then the sensible strategy is to convince most of the participants to not play this game and instead use their energies to prevent all from doing so.

More can be explained by EI Wilson's sociobiology, which says sacrificing one's life is a winning strategy if it leads to survival of portions of your DNA amongst numerous relatives. But machines haven't got any DNA.

In sum, using these tools about 99% of human behavior is explained. The remainder are just nuts. It happens.

Well, this is based on an erroneous statement of the actual theory. It is the survival of the most fit to adapt, not the survival of the fittest. If you're going to run withthe erroneous latter statement, then sure you're probably right, but please don't pin it on Darwin and Wallace. It belongs to nationalists trying to boost support by claiming that they and their nationality/ethnicity are the most fit.

 

[Hornbein]

Well, since this is based on an erroneous statement of the actual theory, we can junk this. It is the survival of the most fit to adapt, not the survival of the fittest. If you're going to run withthe erroneous latter statement, then sure you're probably right, but please don't pin it on Darwin and Wallace. It belongs to nationalists trying to boost support by claiming that they and their nationality/ethnicity are the most fit.

Thank you for sharing.

 

[ShadowKraz]

I find all AGI debates rather strange, because we do not have a common sense definition of AGI itself by now. Then about what are we even discussing then? Conscious AI? Without even knowing or understanding Consciousness? More intelligent as a human? Intelligence also is not quite understood in human. So, about what are we talking exactly?

Exactly. What we're discussing has become a buzzword to sell. "Oh, look, it has AI! yippee! buy now!"

 

[ShadowKraz]

The lack of agency is an interesting argument. In some ways, it's a Catch 22 since people are so afraid of letting them run loose in an unconstrained fashion. How do you have autonomy when you're purposely held back?

This is why I find Moltbook fascinating - it allows them to interact in a less constrained environment. Many of the posts are advertisements and general nonsense but there are occasional posts that have some interesting discussions. Of course there is debate on whether the 'unique' posts are truly unique or just reflections of their human inspired training. More goalpost shifting?

One paper that I always think of when discussions of AGI come up is one that came out just a few months after ChatGPT was released in Nov. 2022 - Generative Agents: Interactive Simulacra of Human Behavior. In it, they gave rudimentary agents an initial description of who they were, a basic memory capability, and full autonomy over their actions with each other in a sandbox environment. The researchers tested their abilty to follow through on their personal 'desires' by occasionally injecting ideas to the agents like "you want to throw a Valentine's Day party" or "you want to run for mayor" and then let them work out for themselves how they would accomplish those goals.

With nothing more than the original ChatGPT LLM (3.5), the agents showed a remarkable ability to decide how to do things like how and where to have their party, and whether or not to attend. They also had discussions with each other about their choices for mayor based on the platforms of the candidates. LLMs have advanced far beyond 3.5 in the three years since but the memory component will still be the critical factor for AGI in general. How a system generates, organizes, retrieves and disposes of memories is the key.

Not just how, but why. Why this piece of data and not that one? A subtle difference but important I think.

 

[Hill]

It is the survival of the most fit to adapt, not the survival of the fittest.

AFAIK, it's rather the survival of the most fit to pass their traits to the next generation.

 

[Hornbein]

AFAIK, it's rather the survival of the most fit to pass their traits to the next generation.

In sociobiology "evolutionary fitness" is usually measured by quantity of third generation offspring, like grandchildren.

 

[Hill]

In sociobiology "evolutionary fitness" is usually measured by quantity of third generation offspring, like grandchildren.

You're right. Perhaps I should've said, "to pass their traits to following generations."

 

[Borg]

Not just how, but why. Why this piece of data and not that one? A subtle difference but important I think.

Yes, there are many things to think about with respect to how a memory system would be built and the questions that would go into building it. Giving an AI the ability to build an maintain its own "world view" has both coding and moral considerations - both of which, have enormous challenges.

 

[ShadowKraz]

AFAIK, it's rather the survival of the most fit to pass their traits to the next generation.

Which would be their adaptability; if they can't adapt, they won't successfully reproduce.

 

[DaveC426913]

Which would be their adaptability; if they can't adapt, they won't successfully reproduce.

Not all species need to adapt in order to pass on their genes.
The world goes through periods of great change, but it also has periods of great stability.

Many creatures have successfully passed on their genes while going through very little adapation over millions or even tens of millions of years.

 

[Hill]

Which would be their adaptability; if they can't adapt, they won't successfully reproduce.

Can't they adapt and live long and happy life but fail to successfully reproduce because no potential mates want them? Or, because potential mates prefer less adapted individuals?
We're not discussing cases when they can vs. can't adapt, but rather when they are more vs. less fit to adapt.

 

[Filip Larsen]

In evolutionary systems (biology included), it is as far as I know common to ascribe the adaptability of the population to the genetic mixing (with mutation playing a lesser role) that happens from generation to generation, and not as much to the adaptability of the individual.

Of course, a population of individuals capable of some degree of adaptation to changes in critical living conditions in parts of their live will of course, everything else being equal, have a higher chance of passing on their genes, but a population as a whole can be adaptable to slow enough changes even if the individuals are not.

 

[ShadowKraz]

Not all species need to adapt in order to pass on their genes.
The world goes through periods of great change, but it also has periods of great stability.

Many creatures have successfully passed on their genes while going through very little adapation over millions or even tens of millions of years.

Which means they have the ability to adapt to changes in their environment already in their genes and aren't under pressure to evolve differently.

 

[ShadowKraz]

Can't they adapt and live long and happy life but fail to successfully reproduce because no potential mates want them? Or, because potential mates prefer less adapted individuals?
We're not discussing cases when they can vs. can't adapt, but rather when they are more vs. less fit to adapt.

Point, thank you. I'll have to re-think.

 

[ShadowKraz]

Can't they adapt and live long and happy life but fail to successfully reproduce because no potential mates want them? Or, because potential mates prefer less adapted individuals?
We're not discussing cases when they can vs. can't adapt, but rather when they are more vs. less fit to adapt.

When talking of survival of the species, as Darwin was, the clue is in the statement... the species. A species with mating regularly occurring between individuals with lower genetic ability to adapt will eventually go extinct as their environment changes. Cheetahs are a case in point; their genetic diversity and, hence, the species ability to adapt to changes, have put the species in danger in evolutionary terms. But, they mate. Hopefully, for the sake of this beautiful species, they will regain some of their initial range and thus increase their genetic diversity.

But moving on... AI is not a species as such. It is a collection of similar in function but unique programs. Can they cross breed? Can they develop the, to use the term a bit loosely, genetics to interbreed and evolve? Or will they always be dependent on human intervention to evolve them?

Also, intelligence needs judgement. Why this bit of data rather than that one? Humans use judgement when accessing and processing new data (although I question our actual ability, as a species, to do so well). AFAIK, all the judgement they have is strictly defined by their programming and admits of no internal adjustment; they don't and can't judge whether or not the data they are accessing and processing is valid. Remember, repetition is not validation. They have to be instructed by humans. This, yes, is akin to rearing a child. As parents, we have to guide our children to make these value judgements by fostering critical thinking. At some point, we have let our children make their own judgements, whether for good or ill. Do AI have the ability to actually learn and synthesize beyond that guidance?