How LLMs work and other things

[jedishrfu]

https://www.wired.com/story/how-chatgpt-works-large-language-model/

The article mentions how Reddit and Stack want to monetize their content wrt LLM usage in their training sets.

What do you think @Greg Bernhardt ?

 

[Grelbr42]

If a social network can monetize my posts, I want my cut.

 

[russ_watters]

If a social network can monetize my posts, I want my cut.

That's not how it works now. You are the product and the social media company makes its money by selling your eyes to advertisers, using your posts (including non public ones) to target the ads. Your "payment" is the free use of the website.

I'm skeptical though that publicly available information can be monetized for LLM training. These posts are public because we want as many people to read them as possible.

 

[russ_watters]

In case others didn't see this:

We're the #1 most represented physics domain in Google C4 dataset!

View attachment 325203

 

[Greg Bernhardt]

What do you think @Greg Bernhardt ?

PF will have a search chatbot trained on our posts at some point, yes. I don't know about monetizing though. Also, Reddit and Stack and many times larger than PF, so they have much more data to work with.

 

[fresh_42]

I don't know about monetizing though.

You should inform yourself quickly!
I have read somewhere that SE went over the counter for billions! Ok, let's be honest. It was only 1.8 billion.

StackExchange, known as StackOverflow, has been sold to Prosus for $1.8 billion. This includes the TeX StackExchange Q&A site (TeX.SE). Prosus is a technology investor and holding company that already owns companies like Udemy, Codecadamy and Brainly (“Your 24/7 homework helper”). After $153 million in investor funds, hiring a former investment banker as head, several rounds of layoffs, moderators leaving, becoming more efficient by standardizing sites, exit speculation, it wasn't really surprising. Official communications and Joel Spolsky's announcement say "business as usual" and that everything will continue as is. So, nothing to see here.

 

[Greg Bernhardt]

I have read somewhere that SE went over the counter for billions! Ok, let's be honest. It was only 1.8 billion.

Pretty aggressive considering LLMs are likely going to put SE out of business.

 

[fresh_42]

Pretty aggressive considering LLMs are likely going to put SE out of business.

Maybe SE but certainly not MO (and possibly other Overflows I don't know of). The discussions there are really on an academic level.

And with regards to SE: yes, they might have more sections than we have, but we are far better than they are.
We do not downgrade users for correct answers, and we do not delete seemingly silly questions as long as the user shows his efforts to come to a conclusion, just to mention two aspects we are better at, despite all users who think we would be unfair. At least we try. SE punishes.

 

[Greg Bernhardt]

Maybe SE but certainly not MO (and possibly other Overflows I don't know of). The discussions there are really on an academic level.

You just wait a couple of years. I'm interested to see what Wolfram Alpha's plugin can do with ChatGPT.

 

[fresh_42]

You just wait a couple of years. I'm interested to see what Wolfram Alpha's plugin can do with ChatGPT.

I would have expected that ChatGPT would analyze at least Wikipedia in general and WA on my specific question

Both are obviously not the case. It failed to check Wikipedia on bismuth, and WA on primes. There is a long way ahead once the general hype has settled down.

 

[Greg Bernhardt]

I would have expected that ChatGPT would analyze at least Wikipedia in general and WA on my specific question
View attachment 325759

Both are obviously not the case. It failed to check Wikipedia on bismuth, and WA on primes. There is a long way ahead once the general hype has settled down.

It can't browse the internet yet (there is an alpha model) but it is trained on a lot of wiki pages. I don't disagree there are issues, but the tech is moving quickly. We'll get there before too long.