The Fakes Keep Getting Better

[Hornbein]

I thought it was real. It had to happen.

 

[Greg Bernhardt]

I think this next generation is going to have a real struggle with reality. Or they just won't care or know to care.

 

[AlexB23]

Yeah, generative artificial intelligence is gonna do a number on society. My hope is that AI generated content could be labeled, or detected as being AI. Or watermarked with something like SynthID, but an open standard.

 

[DaveC426913]

Band member of virtual K-pop group falls pregnant, leaves group

https://www.yahoo.com/entertainment/music/articles/fans-ai-k-pop-band-183000244.html

"The AI-generated band known as BABYQUEEN is a Chinese girl group in the same vein as real-life K-pop acts. The group launched back in September with five virtual members, all of which were accompanied by social media accounts and personas to share their fictitious activities with fans."

the band's fictional management company ... announced that one of the members, Mity, was pregnant and would take maternity leave. The announcement, which was posted to the Chinese social media platform Weibo explained that Mity would be suspending her activities in the band to prioritize her "health" as well as prepare for childbirth.

 

[Greg Bernhardt]

Yeah, generative artificial intelligence is gonna do a number on society. My hope is that AI generated content could be labeled, or detected as being AI. Or watermarked with something like SynthID, but an open standard.

Trouble is there will be too many open weight models that either don't care or can be jailbroken

 

[AlexB23]

Trouble is there will be too many open weight models that either don't care or can be jailbroken

Luckily, many people do not have the hardware to run decent open-weights models. And the open-weights community has been speaking out against using AI for bad. AI is a tool like a car. Can be used for good (point A to B) or bad (driving recklessly).

Open-weights models might even be safer, as they can find critical security flaws that closed models cannot.
https://www.businessinsider.com/hugging-face-ceo-open-weight-ai-models-china-2026-8

OpenAI banned user after he used ChatGPT tool to hack his own TV:
https://www.tomshardware.com/tech-i...nd-plenty-of-other-user-data-for-ad-targeting

 

[pbuk]

I thought it was real.

I thought you played guitar - or have you got a bass that looks like this:

and sometimes this:

and even this:

?!!!

 

[QuarkyMeson]

Is there any reason to actually care? People have been faking video and images since time immemorial.

Trouble is there will be too many open weight models that either don't care or can be jailbroken

Rather than trying to watermark every fake, it would be easier to authenticate original recordings. A camera could cryptographically sign its output using a private key (4 of course), allowing anyone with the corresponding authenticated public key to verify its origin and integrity.

Call it cryptocinematography. It wouldn't prove that the recorded events were genuine, but it would establish a verifiable chain of provenance.

 

[Hornbein]

Is there any reason to actually care? People have been faking video and images since time immemorial.

For entertainment it doesn't matter to me. I don't like false history or fake videos of notable people with words put in their mouths. The advance of the technology is also of interest.

AI has hugely increased the quantity of such efforts. One gentleman distributing phony history was putting out twelve vids a day. This sort of thing has since been banned by Youtube, presumably because YT pays for the storage.