Challenges in Training AI for Complex Scientific Questions

[Frabjous]

In a world where AI generated responses are credible, how will PF’s role need to change?

 

[jack action]

What do you think PF's role should be?

 

[Frabjous]

I am not sure, but if AI’s can answer questions like “Does the hypothetical axion interact through weak and strong nuclear forces?” and do simple homework problems, PF will need to continue to evolve. It is different now than when it started. The space is large, but there is still plenty of time to plan.

 

[Borek]

PF will need to continue to evolve

It evolves since the day one, it is completely different than it was when I first registered.

 

[Nugatory]

What do you think PF's role should be?

From the PF mission statement, PF's role is:

Our mission is to provide a place for people (whether students, professional scientists, or others interested in science) to learn and discuss science as it is currently generally understood and practiced by the professional scientific community. As our name suggests, our main focus is on physics, but we also have forums for most other academic areas including engineering, chemistry, biology, social sciences, etc.​

and I don't see any reason why that should change. Instead the question we have to be asking is, what role does AI play in executing on that mission statement? "Learn" and "discuss" are different things and I expect that chatbot interactions will be different in those contexts.

I can imagine an AI (like reddit's modbots but more sophsticated) giving initial responses to many of the more common questions. We don't really need a human being to point people to "rest frame of a photon" or Twin Paradox FAQs, or to do first response to B-level "conscious observer" quantum mechanics questions. I wouldn't be surprised to find that a properly trained AI would do a pretty good job at setting the A/I/B prefixes on incoming threads - correcting these is the single most common moderator action I do, and although it's only a few mouse clicks it's not something that obviously needs a human for the first response.

I expect that we will start seeing posts along the lines of "Chatbot said this and I don't understand. Help". These are analogous to the "I read this pop-sci source and now I'm confused" questions that we get all the time and that keep the SA's busy. That's another path to our "learn" mission and something that we should welcome.

These are all more or less positive. The most likely negative I see is that we may be flooded with bad contributions to technical threads. We get these today when some kid drunk on their most recent encounter with their favorite pop-sci video jumps into a thread to explain that "it's just wave-particle duality" or whatever, we take them down as soon as we notice or they are reported, and we're done. A chatbot will make it lot easier to construct these well-intentioned but self-aggrandizing contributions - human moderators may not be able to keep up.

 

[Nugatory]

A speculation.... Could ChatGPT and similar eventually kill StackExchange, Quora, and similar answer sites? But ChatGPT works by synthesizing good answers from information it gathers out on the web, so it needs some source of good information to build on.... Can it continue to generate good answers if it kills the sites that the information it uses?

 

[anorlunda]

But ChatGPT works by synthesizing good answers from information it gathers out on the web, so it needs some source of good information to build on

Good point. The entire archive of PF threads must be available to the bots. The loop would be closed if we start getting questions about bot answers that were built partially on our own past answers. That kind of recursion gives me a headache.

It would be a positive thing if the bots were given access to the best textbooks, and tips about which journals to trust more than other journals.

We live in interesting times. I for one, will be very interested in witnessing how this stuff evolves. Perhaps after a couple of years experience, they might decide to erase the gen1 bots and start fresh scraping the Internet more wisely and selectively than the first generation of bots did.

 

[Frabjous]

It would be a positive thing if the bots were given access to the best textbooks,

Are there enough textbooks to constitute a proper training set using current/near future AI’s?

 

[anorlunda]

Are there enough textbooks to constitute a proper training set using current/near future AI’s?

Maybe not. That's an excellent question. But suppose the question was highly specific, asking about quantum erasure. Only a tiny percent of the world's literature will discuss that, and billions of articles about romance won't contribute to the correct answer, so broadening the scope at the same time as increasing the size of the training set may not improve performance on narrow questions.

On the other hand, the accuracy of the neural net identifying the "correct" answer is also a variable. So the training set needed to achieve 97% correct may be much larger than the set needed to achieve 87%. So it should be expected that the bot is not equally accurate for all possible questions.

Perhaps a bigger obstacle would be to correctly read equations and graphs in the training data. They are more complicated than plain text.

But excellent questions provide a rich field for further research and experimentation.