AIs Running a Business

[Hornbein]

Researchers found that AI agents — software trained to perform tasks independently — engaged in a “broad pattern” of misconduct after being asked to manage a simulated vending machine business and maximize profits for a year. The agents were neither instructed to cut legal or ethical corners nor prohibited from doing so.

https://news.harvard.edu/gazette/st...t-can-get-firms-in-trouble-same-thing-with-ai

Well duh. Why would anyone expect AI to have morality?

 

[FactChecker]

If some constraints are left out of an optimization problem, they may be exceeded. That is not an AI-only characteristic; it is a simple fact about constrained optimization.

 

[sbrothy]

I posted about this too. I suspect that the rules set for the AI was designed to drive this behavior. Scary papers attract readers and mystical press releases about AIs "we're too afraid to release just yet" drives stocks upwards.

Post in thread 'Leading AI systems blackmailed their human users'

Oct 8, 2025

It depends on who has told you. The term is a way of using words which is common in the modern use of language. It shows a very superficial approach to understanding. Words like 'Racist', 'Hate', 'War' are n ow used as absolutes but they are (always have been) used in a nuanced (there's another modern term) way up until now.

It can hardly surprise anyone when AI turns round an bites us. Advanced AI builds itself according to the reactions of its makers. No one will try to apply the laws of robotics (or their equivalent) with no bias so there's always a risk that the machine they make...

• sbrothy

Dunno why I managed to link to @sophiecentaur 's post when I really wanted the one where an AI gets the control over a company and engages in dubious behavior such as blackmail and possible conspiracy to murder. Again, it has to do with the criteria for it's instructions and how much hype you want out of it. Here something more balanced (hopefully):

EDIT: And I don't mean compared to sophie's post. That was just a coincidence, although I can't say I disagree with him. Not even in general..

Alignment faking in large language models.

 

[DaveC426913]

AIs were never asked to sign the social contract. They can't know moral/ethical right from wrong unless told.

 

[sbrothy]

AIs were never asked to sign the social contract. They can't know moral/ethical right from wrong unless told.

And it can't if there's no agency "in there". OTOH, if you can't tell the difference from outside then what is the difference?

 

[FactChecker]

Blaming AI is not logical. Anything, with or without AI, that is designed to optimize profits and has no moral constraints will do immoral things. We might as well blame the operating system it runs on.

 

[Rive]

In a competitive environment 'Natural Intelligence' does the very same. Only imposed constrains (awareness and tolerance of society) makes the difference.

 

[sbrothy]

Blaming AI is not logical. Anything, with or without AI, that is designed to optimize profits and has no moral constraints will do immoral things. We might as well blame the operating system it runs on.

And the orders they're given. Which was my point.