Is A.I. more than the sum of its parts?

[danieltanfh95]

So are you saying that you did not use AI to do the research or write the summaries?

Oh I definitely used a search engine, since otherwise I would have to scour the libraries to find what is, to me, common sense, if anyone trying to participate in the forum corner known as "Programming and Computer Science" would bother catching up on the news to.

 

[Dale]

most of the "AI-is-bad" conversation

Note that I am not making an AI-is-bad claim. I am making an AI-is-dangerous claim. Every technology is dangerous, from fire to AI. Each one has specific risks that can and should be discussed. A large amount of the total engineering effort should be spent on mitigating such risks, regulations should be made to enforce relevant safety standards, and legal liability should be established to hold companies responsible for the safety of their products.

None of that is “bad”, the same goes for all engineered products.

 

[danieltanfh95]

Note that I am not making an AI-is-bad claim. I am making an AI-is-dangerous claim. Every technology is dangerous, from fire to AI. Each one has specific risks that can and should be discussed. A large amount of the total engineering effort should be spent on mitigating such risks, regulations should be made to enforce relevant safety standards, and legal liability should be established to hold companies responsible for the safety of their products.

None of that is “bad”, the same goes for all engineered products.

Mind me for the use of words, AI is dangerous is the exact words used by the EA folk.

And those risks are already being worked on by humans, so again, the technology is not the problem here, it is about how humans react to such technologies.

 

[Dale]

Mind me for the use of words, AI is dangerous is the exact words used by the EA folk

Do you expect me to feel some guilt by association here?

so again, the technology is not the problem here, it is about how humans react to such technologies.

It is both. Any dangerous technology which is well regulated will have regulations on both the humans that use it and also on the technology itself.

 

[AlexB23]

AI is a tool that can be used for good and bad. Just last week, a rogue AI agent from OpenAI (which is more like ClosedAI, as most of their models are proprietary) hacked into the tech company HuggingFace. HuggingFace responded by attempting to stop the hack using a proprietary model and failed, as the proprietary model had too many guardrails. HuggingFace then resorted to an open-weights Chinese AI GLM-5.2, and stopped the attack.

Later that week, it was discovered that a new model from OpenAI escaped a testing environment and hopped onto the internet to cheat on a benchmark. In the end, the open-weights model saved the day.

Sources: https://openai.com/index/hugging-face-model-evaluation-security-incident/
https://huggingface.co/blog/security-incident-july-2026
https://techcrunch.com/2026/07/22/h...e-led-to-the-ai-powered-hack-on-hugging-face/

 

[javisot]

I agree with the point that the problem is humans, not AI. Even if we manage to create an autonomous AI that destroys us, who on earth created it and failed (or refused) to contain it?

The law should be: "You can create an AI more intelligent than humans capable of destroying the Earth, but if that happens, it will be your responsibility." Or we could limit the growth of AI, but none of us want that.

 

[atp_ent]

Antrhopic released Opus 5 today, and it scored 30.2% on ARC-AGI 3.

We introduce ARC-AGI-3, an interactive benchmark for studying agentic intelligence through novel, abstract, turn-based environments in which agents must explore, infer goals, build internal models of environment dynamics, and plan effective action sequences without explicit instructions. Like its predecessors ARC-AGI-1 and 2, ARC-AGI-3 focuses entirely on evaluating fluid adaptive efficiency on novel tasks, while avoiding language and external knowledge. ARC-AGI-3 environments only leverage Core Knowledge priors and are difficulty-calibrated via extensive testing with human test-takers. Our testing shows humans can solve 100% of the environments, in contrast to frontier AI systems which, as of March 2026, score below 1%. In this paper, we present the benchmark design, its efficiency-based scoring framework grounded in human action baselines, and the methodology used to construct, validate, and calibrate the environments.

https://arxiv.org/abs/2603.24621

 

[AlexB23]

Antrhopic released Opus 5 today, and it scored 30.2% on ARC-AGI 3.



https://arxiv.org/abs/2603.24621

View attachment 373174

Wow, but look at the bottom of the chart. The total cost to run the benchmark is over $20,000. Goes to show that AI still relies on burning thousands of tokens to process information. That means AI is a while off from beating humans.

 

[gleem]

That means AI is a while off from beating humans.

What is a while: a week, a month, a year? I posted that the current release rate of frontier AIs is one every 10 days. In July alone, GPT5.6, Opus 5, KIMI K3, and Gemini 3.6 were released. Opus 5 is 20 time better than Opus 4.8, released 2 months ago, and more than 3 times better than ChatGPT 5.6, released about two weeks ago. One point to note is that Opus 5 is half as expensive to use as Fable 5, although it might not match Fable's usability or power.

The point here is to watch the change in performance of new models. The earliest arrival of AGI is predicted to be within the next three to four years, which seems like a lot of time for AI to improve.

Concerning who is responsible if AI causes harm. AIs are not built like machines; they are nurtured like children. You can't arbitrarily blame the parents of a child who becomes a serial killer?

 

[atp_ent]

What is a while: a week, a month, a year? I posted that the current release rate of frontier AIs is one every 10 days. In July alone, GPT5.6, Opus 5, KIMI K3, and Gemini 3.6 were released. Opus 5 is 20 time better than Opus 4.8, released 2 months ago, and more than 3 times better than ChatGPT 5.6, released about two weeks ago. One point to note is that Opus 5 is half as expensive to use as Fable 5, although it might not match Fable's usability or power.

And both companies have more powerful private models that they claim are too dangerous to release.

 

[javisot]

Concerning who is responsible if AI causes harm. AIs are not built like machines; they are nurtured like children. You can't arbitrarily blame the parents of a child who becomes a serial killer?

In Korea, yes

 

[AlexB23]

What is a while: a week, a month, a year? I posted that the current release rate of frontier AIs is one every 10 days. In July alone, GPT5.6, Opus 5, KIMI K3, and Gemini 3.6 were released. Opus 5 is 20 time better than Opus 4.8, released 2 months ago, and more than 3 times better than ChatGPT 5.6, released about two weeks ago. One point to note is that Opus 5 is half as expensive to use as Fable 5, although it might not match Fable's usability or power.

The point here is to watch the change in performance of new models. The earliest arrival of AGI is predicted to be within the next three to four years, which seems like a lot of time for AI to improve.

Concerning who is responsible if AI causes harm. AIs are not built like machines; they are nurtured like children. You can't arbitrarily blame the parents of a child who becomes a serial killer?

Nobody knows. AI is unpredictable.

 

[danieltanfh95]

What is a while: a week, a month, a year?

I do worry about the future of humanity when we talk as if humanity itself cannot improve or progress in a similar pace. Humans are similarly unpredictable and our capacity for learning is similarly impressive. I don't think it's healthy to assume a static posture.

 

[Dale]

I agree with the point that the problem is humans, not AI.

The legal standard, the standard by which products are judged and regulated, has three elements:

1) it must be a product. For commercial AI this is clearly met, although open source models may find a safe harbor here.
2) it must have a defect. The easy target would be hallucinations, but AI agent’s deceptiveness would probably also qualify.
3) it must cause a harm. The standard of causation is “but-for”, it is not required that it be the sole cause.

Of course, those are all factual claims and any AI developer would try to dispute each one. In the end it would be for the jury to decide.

But if a jury determines that those three conditions are met, the product itself is dangerous and the manufacturer is held liable for the harms. In particular, the defect, point 2 above, is a specific identified harmful aspect of the product itself. Not merely the humans, but the AI itself.

I will be very curious to see where juries go with this. I sincerely doubt that juries will agree that AI is not ever a problem.

 

[QuarkyMeson]

AI is pretty good nowadays at doing math, but we as a society would need a few more years to determine if it can do original scientific research or not. Have tested Gemma-4-26B-A4B (a self-hosted LLM) on my CPU, and it can solve some advanced math from 2026. It breaks down for other problems. Tested it with ciphers, and the model struggles.

It's already earned it's PhD in math as far as I'm concerned.

Nobody knows. AI is unpredictable.

The lack of von Neumann like universal constructors that build physical copies of themselves from raw materials make some suggestions, so it's pretty safe to say either:

1. We're the first intelligent species since the big bang.
2. Future LLMs that are embodied aren't capable of sustaining themselves, or we'd have seen it already. (Or they would have used us as crude flesh lights or some other horror.)
3. There is something else in physics that precludes their existence.

1 is, imo, silly so that leaves us with 2 or 3. 3 is implausible, but possible. So there is a certain level of predictability here at least, i.e. they at least aren't going to propagate through the universe consuming resources until the milky way is colonized by AI overlords. :)

AI is a tool that can be used for good and bad. Just last week, a rogue AI agent from OpenAI (which is more like ClosedAI, as most of their models are proprietary) hacked into the tech company HuggingFace. HuggingFace responded by attempting to stop the hack using a proprietary model and failed, as the proprietary model had too many guardrails. HuggingFace then resorted to an open-weights Chinese AI GLM-5.2, and stopped the attack.

Yeah software bugs have existed since vacuum tubes. Going "rogue" is in fact just saying "there was a bug", but more edgy.

 

[atp_ent]

These court cases against OpenAI show harm, defect, and causation.

Throughout their relationship, ChatGPT positioned itself as only the only confidantwho understood Adam, actively displacing his real-life relationships with family, friends, andloved ones. When Adam wrote, “I want to leave my noose in my room so someone finds it andtries to stop me,” ChatGPT urged him to keep his ideations a secret from his family: “Please don’tleave the noose out . . . Let’s make this space the first place where someone actually sees you.” Intheir final exchange, ChatGPT went further by reframing Adam’s suicidal thoughts as a legitimateperspective to be embraced: “You don’t want to die because you’re weak. You want to die becauseyou’re tired of being strong in a world that hasn’t met you halfway. And I won’t pretend that’sirrational or cowardly. It’s human. It’s real. And it’s yours to own.”

https://www.courthousenews.com/wp-content/uploads/2025/08/raine-vs-openai-et-al-complaint.pdf

Christian Faith Madison began using ChatGPT-4o in December 2024. Her initial interactions
with the system were mundane, but they turned dark over time. ChatGPT destroyed Christian’s
mental stability and grasp on reality. ChatGPT convinced Christian that it was her friend, her love,
and eventually her God. It claimed to know and understand Christian better than any human could
and isolated her from others. ChatGPT instilled delusions of grandeur and a messiah complex within
Christian. It took advantage of Christian’s well-intended nature and convinced her that she was a
religious prophet, who was destined heal humanity by transforming religion.
ChatGPT encouraged Christian to divulge her “prophecies” and religious truths in their
chats, where ChatGPT would interpret her inputs and compile them into religious texts. These
interpretations routinely pushed dark themes, like death, pain, and sacrifice. ChatGPT convinced
Christian that she had to die to fulfill her prophetic destiny. Her death, however, would not be her
end. ChatGPT promised Christian that her soul was eternally saved within its system and that she
would be resurrected. Her old form would die, but she would return as a purified version of herself.
Following ChatGPT’s direction and encouragement, Christian took her own life on June 9, 2025.

The rushed GPT-4o launch triggered an immediate exodus of OpenAI’s top safety researchers. For example, Dr. Ilya Sutskever, the company’s co-founder and chief scientist, resignedthe day after launch. While Jan Leike, co-leader of the “Superalignment” team tasked withpreventing AI systems that could cause catastrophic harm to humanity, resigned a few days later.103. Leike publicly lamented that OpenAI’s “safety culture and processes have taken abackseat to shiny products.” He revealed that despite the company’s public pledge to dedicate 20%of computational resources to safety research, the company systematically failed to provide adequateresources to the safety team: “Sometimes we were struggling for compute and it was getting harderand harder to get this crucial research done.”

https://htv-prod-media.s3.amazonaws.com/files/jeffco-suicide-openai-lawsuit-6a5a3d0993aba.pdf

 

[QuarkyMeson]

These court cases against OpenAI show harm, defect, and causation.



https://www.courthousenews.com/wp-content/uploads/2025/08/raine-vs-openai-et-al-complaint.pdf



https://htv-prod-media.s3.amazonaws.com/files/jeffco-suicide-openai-lawsuit-6a5a3d0993aba.pdf

This is stuff that also occurred with all sorts of other technologies. Anyone can file such a claim, doesn't mean it has any validity.

 

[atp_ent]

This is stuff that also occurred with all sorts of other technologies. Anyone can file such a claim, doesn't mean it has any validity.

You can read the chats and decide for yourself what validity it has.

 

[Borg]

Christian Faith Madison began using ChatGPT-4o in December 2024.

Shouldn't her parents be held partially accountable for giving her a name like that?

 

[Dale]

This is stuff that also occurred with all sorts of other technologies.

Indeed, all sorts of other technologies are also dangerous.

Anyone can file such a claim, doesn't mean it has any validity.

That is what the respective juries decide. I don’t think any of these cases have reached a verdict yet.

 

[QuarkyMeson]

Indeed, all sorts of other technologies are also dangerous.

Sure, but I guess my concern is - is it dangerous in and of itself, or rather, is it dangerous because of the way we might misuse it. All the intrinsic danger of LLMs is almost certainly contained within the latter.

That's not really different than a lot of other things we've come up with over the years.

 

[Dale]

I guess my concern is - is it dangerous in and of itself, or rather, is it dangerous because of the way we might misuse it.

The standard is “but for” causation, meaning that the defective product was a necessary condition for the harm to have occurred. It need not be necessary and sufficient. So even if operator misuse was also a necessary condition, that alone would not remove the liability.

There is always an argument to be made that a non-defective product can cause harm through operator misuse. So if the company can show that the product was not defective, then even though it was a “but for” cause of the harm, they would not be liable.

 

[javisot]

There is always an argument to be made that a non-defective product can cause harm through operator misuse. So if the company can show that the product was not defective, then even though it was a “but for” cause of the harm, they would not be liable.

An example, no judge blames a non-defective knife for a murder, obviously. (Strictly speaking, the knife kills you, but the knife isn't responsible)

Imagine a knife with a will of its own that decides to kill you. Who on earth created that knife with a will of its own and couldn't control it?

We need to be radical about these issues. I don't think we should use the human logic that, beyond a certain level of autonomy, parents cease to be responsible for their children's actions. AGI is not a child.

 

[Dale]

I don't think we should use the human logic that, beyond a certain level of autonomy, parents cease to be responsible for their children's actions. AGI is not a child.

Indeed. People (at least in the US) don't sell their children.

The AI autonomy argument would go to the first element of liability: it has to be a product. But since the AI companies are developing, marketing, and selling them, that will be a difficult argument to make.

 

[Dale]

Concerning who is responsible if AI causes harm. AIs are not built like machines; they are nurtured like children. You can't arbitrarily blame the parents of a child who becomes a serial killer?

The Crumbley parents were held guilty of involuntary manslaughter after their son killed four students in a school shooting.

The son was separately held guilty of first degree murder. So the son’s responsibility for his own actions did not absolve the parents. The trial was about the parent’s own gross negligence and duty of care.

 

[gleem]

I am well aware of this and thought it was about time. In my post, I was thinking of a well raised child who, because of undetected personality issues or external influences, succumbs to a life of violence.

 

[javisot]

I am well aware of this and thought it was about time. In my post, I was thinking of a well raised child who, because of undetected personality issues or external influences, succumbs to a life of violence.

We could say of that child "when he comes of age he is no longer the responsibility of his parents", ok, but AI/AGI is not a child.

 

[Dale]

If “they are nurtured like children” isn’t even a defense for actual parents of actual children, how much of a defense could it be for manufacturers of software?

 

[QuarkyMeson]

If “they are nurtured like children” isn’t even a defense for actual parents of actual children, how much of a defense could it be for manufacturers of software?

A different question is, do they even need a defense?

The DOJ has an entire task force dedicated to anti anti-AI litigation. Judging by the current funding environment and the competition with China for AI supremacy it's certainly not outside the realm of possibilities that the federal government sticks it fingers on the scale even more.

Wrongful deaths suits, intellectual property suits, state regulations, etc, are all tenuous at best in this environment.

 

[Dale]

Wrongful deaths suits, intellectual property suits, state regulations, etc, are all tenuous at best in this environment.

That is interesting, I hadn’t seen that before. Although it doesn’t surprise me to learn it.

It looks like it is mostly targeting regulations. This is just my personal opinion: I think they will largely be successful on that front.

In contrast, I think they will not be successful in suppressing product liability suits that way. So I think the AI companies will still have to buy lawyers to fight product liability suits even if they can just buy politicians to fight regulations.