Discussion of PF Policy on Valid References

[AndreasC]

[Moderator's Note: Spin off from previous thread due to topic/forum change.]

This is a pop science book. You would be better served by reading actual textbooks or peer-reviewed papers.

That's honestly sort of a weird reply but it's an attitude I've encountered a lot on this site. There is a reason pop science exists. Say you hear about a major discovery in some branch of science you know nothing about. What do you do to learn more about it? There is 3 options. You either start reading papers (which is a surefire way to not understand anything), you read pop-sci accounts, or you chose to ignore it completely since you won't "properly" understand (which I can't see as preferable).

If OP is reading pop-sci books, it's probably because they don't understand the legit textbooks, let alone papers.

"But it is an I-level thread". Well ok but most undergrads can't read complicated papers on these advanced subjects, and who knows if OP is even an undergrad. The B, I, A system on this site is a bit weird sometimes because really only "A-level" people can usually judge correctly what the level of a subject is, and the rest are often not sure whether to use what describes themselves or what describes the subject. For instance maybe a beginner wants to learn something about string theory, but string theory is an A level topic, so what do they chose?

My point is that I'm not even sure OP is really I-level, and even if they are it doesn't mean they can just read the papers and understand, and they shouldn't be blamed for reading pop-sci. And honestly I don't believe people should not be allowed to ask questions on subjects that are supposedly beyond their level.

[Moderator's note: Comment relevant only in context of original thread has been removed.]

 

[PeterDonis]

There is a reason pop science exists.

Yes, we know that. And there are also good reasons why pop science is not considered a valid basis for discussion in our science forums.

If OP is reading pop-sci books, it's probably because they don't understand the legit textbooks, let alone papers.

And if that is the case, they won't be able to understand actual science. Sorry, but that's just the way it is.

most undergrads can't read complicated papers on these advanced subjects

Many peer-reviewed papers are really "A" level, yes--they assume the reader already has a graduate level understanding of the subject matter. One of the main functions textbooks serve is to fill the gap by giving "I" level (or in some cases "B" level) presentations of subjects.

However, since science is an ongoing enterprise, there are many areas of current research that textbooks do not cover, so the only possible sources are peer-reviewed papers. That means either you build up your understanding so you can follow at least a portion of what the papers are saying, or you won't be able to understand that area of research. Again, sorry, but that's just the way it is.

For instance maybe a beginner wants to learn something about string theory, but string theory is an A level topic, so what do they chose?

Either they build up their own understanding of the relevant subject areas to "A" level, or they accept that whatever they "learn" about string theory will not be the actual science of string theory. Maybe that's ok with them: maybe all they really wanted to know about string theory was something like "string theory is a quantum field theory based on strings instead of point particles", which is a true statement as far as it goes but tells you nothing useful unless you already know something about quantum field theory, which is also generally considered an "A" level subject. Or maybe they really just want to ask something like "is string theory a valid theory of everything?", in which case the answer would be "many of its proponents claim it is, but at present there is no valid basis for the claim". Which, again, doens't really tell you anything about the science of string theory, but the question wasn't really about that anyway.

I don't believe people should not be allowed to ask questions on subjects that are supposedly beyond their level.

Nobody is saying that. All I am saying is that it is simply a fact of life that you can't learn actual science from pop science sources. Pop science sources do not magically make "A" level subject matter into "I" or "B" level subject matter. Pop science sources can be very good for getting people interested in particular areas of scientific research. They just are not good for actually learning the actual science.

The only way to learn the actual science is by looking at the actual science. The primary sources for actual science are peer-reviewed papers, so if you really want to see the actual science, that's what you need to look at. Textbooks can be useful secondary sources for areas that are established enough to have good textbook treatments. Textbook treatments are more likely to be "I" level (or in some cases even "B" level), if such a treatment is possible for the subject matter.

If you ask a question about an ongoing area of current research, it's quite possible that the only meaningful answers available will be "A" level answers. Again, that's just a fact of life: nobody can magically make "A" level subject matter into "I" or "B" level subject matter. So you might not be able to really understand the answer. But if that happens, saying "well, just look at a pop science source" won't help anything.

 

[AndreasC]

And if that is the case, they won't be able to understand actual science.

It's perfectly valid for people to want to get a level appropriate, intuitive general picture of things to the extent that it is possible. Maybe sometimes it's just not possible, but here I don't think that's the case.

I don't see what's wrong with someone coming here and saying "I read this in pop-sci, what does it mean?" or "what more is there to this?". Worst case, it's something completely wrong so you can tell them it's completely wrong.

Either they build up their own understanding of the relevant subject areas to "A" level, or they accept that whatever they "learn" about string theory will not be the actual science of string theory. Maybe that's ok with them: maybe all they really wanted to know about string theory was something like "string theory is a quantum field theory based on strings instead of point particles", which is a true statement as far as it goes but tells you nothing useful unless you already know something about quantum field theory, which is also generally considered an "A" level subject. Or maybe they really just want to ask something like "is string theory a valid theory of everything?", in which case the answer would be "many of its proponents claim it is, but at present there is no valid basis for the claim". Which, again, doens't really tell you anything about the science of string theory, but the question wasn't really about that anyway.

Well there you go then. You just answered some questions about string theory in a way that people can understand without having to read Polchinski! Yes, it's not the "actual science", but as you said, most people who read pop-sci aren't interested in the "actual science", and if they are, it's out of reach. But that doesn't mean they can't get answers, albeit in a "not-actual-science" manner. People shouldn't have to learn all the intricacies of convection, conduction, radiation, planetary climate models and whatnot to know that if you pump the atmosphere full of CO2 bad things happen. If they are really interested in the "actual science", then maybe they can eventually pick up a textbook, but it can't wait until then!

 

[PeterDonis]

I don't see what's wrong with someone coming here and saying "I read this in pop-sci, what does it mean?" or "what more is there to this?"

That's because you don't have to moderate the forums. For every question along these lines that's actually coming from a pop science source that gives reasonable (if limited) information, there are a hundred or more that are based on sources that are so off base that it's not even worth trying to answer the question. That's why our standard answer to all such questions is "if you're really interested, do the work to find a valid reference". Often just looking at the references on the Wikipedia page for a subject will give you multiple valid sources.

It's possible, of course, as I've already said, that the subject area is an ongoing area of research so the only valid sources are "A" level sources. But again, in cases like that going to a pop science source instead won't help--areas of ongoing research, where there isn't yet a single established theoretical model that is mainstream, are also areas where pop science sources are the worst and the most misleading. In such cases, if you aren't willing to build up your own understanding to the point where you can at least follow some of the "A" level papers in the field, you simply won't be able to understand that area of research. There is no way to "fix" that--certainly pestering PF to allow discussions based on pop science sources won't do it.

You just answered some questions about string theory in a way that people can understand without having to read Polchinski!

And the answers I gave you were useless if you actually want to learn what string theory says, let alone try to judge for yourself the many extravagant claims that string theorists make. If you are happy with that answer, that's fine, but that just means you don't actually want to learn anything useful.

PF's mission is not to give half-baked useless answers. PF's mission is to help people understand mainstream science, and, to the extent possible, to help people understand at least a portion of what is going on in ongoing areas of research (that's why we have, for example, the Beyond the Standard Model forum). String theory falls in the latter category. As does the topic of this thread.

 

[Vanadium 50]

I don't see what's wrong with someone coming here and saying "I read this in pop-sci, what does it mean?

The problem is when people respond to correct explanations with "You're wrong! I read something else in a pop-sci book."

Another problem is that people try and come up with a broader model by stitching together multiple incompatible pop-science explanations. That never works.

 

[AndreasC]

The problem is when people respond to correct explanations with "You're wrong! I read something else in a pop-sci book."

Another problem is that people try and come up with a broader model by stitching together multiple incompatible pop-science explanations. That never works.

Well, sure. But that just sounds like people being confidently wrong, not an honest question prompted by pop-sci....

 

[PeterDonis]

that just sounds like people being confidently wrong, not an honest question prompted by pop-sci....

It's neither: it's people mistakenly believing that reading pop science allows them to actually understand the science.

 

[AndreasC]

And the answers I gave you were useless if you actually want to learn what string theory says, let alone try to judge for yourself the many extravagant claims that string theorists make. If you are happy with that answer, that's fine, but that just means you don't actually want to learn anything useful.

It seems to me like you believe the only "useful" kind of knowledge is the full weight of detail and formalism. In my opinion this kind of thing just makes sure more or less everything in science is completely inaccessible not just to people outside science, but also scientists working in slightly different branches.

There is certainly a lot of bad pop-sci, but there is also good pop-sci and good "half baked answers" which don't just help beginners see what the fuss is all about, but can even help those who are more advanced gain better intuition, and a "big picture" view, even though they don't directly reference the "actual science".

I don't have to know graduate level biology to learn they found a new frog. I don't have to know advanced genetics to learn they cloned a sheep. I don't have to know the nitty gritty of climate models to learn climate change is bad and caused by humans. I would have to know all these things if I wanted to gain a proper understanding and especially if I wanted to contribute. But that doesn't mean I didn't learn anything useful or interesting.

 

[PeterDonis]

It seems to me like you believe the only "useful" kind of knowledge is the full weight of detail and formalism.

Useful in the sense of actually understanding the science, yes. That's what PF's mission is: to help people understand science. If that's not what you're after, then posting here will most likely not meet your needs.

there is also good pop-sci and good "half baked answers" which don't just help beginners see what the fuss is all about, but can even help those who are more advanced gain better intuition, and a "big picture" view, even though they don't directly reference the "actual science".

I challenge you to give a specific example.

I don't have to know graduate level biology to learn they found a new frog. I don't have to know advanced genetics to learn they cloned a sheep.

Neither of these things are understanding science. They're just factoids.

I don't have to know the nitty gritty of climate models to learn climate change is bad and caused by humans.

Neither of these things are true as you state them. What is true is that some impacts of climate change are bad (in the sense that humans have to pay significant costs to deal with them, for example sea level rise), and that a portion of the climate change we observe is caused by human activities. But there are also impacts of climate change that are good (for example, the current warming is expanding the land area on Earth that is suitable for human habitation and food production), and there are significant causal factors involved with climate change that have nothing to do with human activities.

So if what I quoted above is what you learned from reading pop science, then you have given an example of why pop science is not a good source to learn from.

 

[gentzen]

there is also good pop-sci and good "half baked answers" which don't just help beginners see what the fuss is all about, but can even help those who are more advanced gain better intuition, and a "big picture" view, even though they don't directly reference the "actual science".

I challenge you to give a specific example.

How about
Quantum Chance by Nicolas Gisin
or
The Art of Statistics by David Spiegelhalter

 

[PeterDonis]

How about
Quantum Chance by Nicolas Gisin
or
The Art of Statistics by David Spiegelhalter

How do these help those who are more advanced?

 

[PeterDonis]

help beginners see what the fuss is all about

I have seen cases here at PF where, in response to a "B" level (usually) thread where the OP is really just asking "what's all the fuss about", a good pop science book has been given as suggested reading. If you want to say that is an edge case of "understanding", I won't object.

 

[gentzen]

How do these help those who are more advanced?

The Art of Statistics by David Spiegelhalter
helps those more advanced by providing the bigger picture, also including visualization and communication of results, and of course also emphasizing the importance of the entire problem solving cycle, including planing of experiments, data collection, ...

Quantum Chance by Nicolas Gisin
helps those more advanced by focusing on one specific aspect which is especially counter-intuitive, and illuminates this aspect with intuitive pictures and sharp analyses. It also sets this aspect into its historical context, and hints at some of its possible future developments.

 

[AndreasC]

On that subject, the math channel 3Blue1Brown makes videos giving basic explanations of math concepts which are equally if not more useful for people who already learned the topics elsewhere. For instance I had learned linear algebra but I didn't know until his video that you could think about determinants as volumes, which was extremely useful.

 

[PeterDonis]

the math channel 3Blue1Brown makes videos giving basic explanations of math concepts

I would not classify these as "pop science" sources. I would classify them as attempts to make Internet-era equivalents of basic textbooks, taking advantage of all the technologies we now have for doing pedagogy in different ways.

 

[PeterDonis]

helps those more advanced by providing the bigger picture, also including visualization and communication of results, and of course also emphasizing the importance of the entire problem solving cycle, including planing of experiments, data collection, ...

Wouldn't those more advanced already know these things?

helps those more advanced by focusing on one specific aspect which is especially counter-intuitive, and illuminates this aspect with intuitive pictures and sharp analyses.

Wouldn't those more advanced already understand this aspect (I assume you mean nonlocality)?

It also sets this aspect into its historical context, and hints at some of its possible future developments.

These can be useful but they're not the same as understanding the subject itself as it is.

 

[gentzen]

helps those more advanced by providing the bigger picture, also including visualization and communication of results, and of course also emphasizing the importance of the entire problem solving cycle, including planing of experiments, data collection, ...

Wouldn't those more advanced already know these things?

I don't know. I didn't knew those before, or at least I didn't prioritized them appropriately. But of course, this only shows that I am not sufficiently advanced in this subject. But I had three one-semester courses about such topics at university, so I am not a complete beginner in this subject either.
But probably your other "objection" has even more force against Spiegelhalter:

I would not classify these as "pop science" sources

Wouldn't those more advanced already understand this aspect (I assume you mean nonlocality)?

Yes, of course. But by reading this book, their understanding of this specific aspect might still become more rounded and complete. They might find it easier to explain it to those less advanced after reading this book.

 

[Hornbein]

The "it can't be explained" thesis seems to me often a rationalization for failure. My view is that the ability to explain esoteric things in a way that is both accurate and accessible is quite rare but nevertheless in some cases exists. I found Yau's book on string theory a good example. But what do I know? Maybe he's full of beans and I'm too naive to know it, but I have faith in Yau.

I find the "go away and come back when you have a master's degree in physics" unrealistic. Though being helpful is apparently not the goal with that. Whatever might the goal be?

I find perfectly acceptable the "I can't explain it but I'm a credentialed expert so take my word for it" stance. Though Michio Kaku stands as a counterexample, it is usually the best option. If readers won't accept that and persist in their pestering, to sci.physics with them.

 

[AndreasC]

Hmmm, I don't see why the title is about the valid reference policy. I don't think anyone on this thread argued pop sci should be regarded a valid reference to back up your point or answer or whatever. Personally, I was arguing that it's fine for people to read pop sci and come here with questions prompted by it, and that when they clearly can't possibly understand original literature or advanced textbooks, it should be at least attempted to give a more basic, level appropriate, "pop-sci-like" digested explanation. Of course you can't cite pop sci to back up a claim, but that's a different thing.

 

[PeterDonis]

I was arguing that it's fine for people to read pop sci and come here with questions prompted by it

And that means you are arguing for pop sci sources to be considered valid references for a PF discussion. So you are arguing for PF to change its current policy, which is that pop sci sources are not valid references for PF discussion.

 

[PeterDonis]

I find perfectly acceptable the "I can't explain it but I'm a credentialed expert so take my word for it" stance.

In other words, you find arguments from authority acceptable. But that is not how science is supposed to work, nor is it a way to get actual understanding of the subject matter. If you believe something simply because some authority told you, you don't understand it. PF's mission is to help people understand mainstream science, not to make assertions from authority.

 

[PeterDonis]

I find the "go away and come back when you have a master's degree in physics" unrealistic.

But you find it realistic to read a book on string theory and believe everything it says even though, by your own admission, you don't have the background knowledge to judge anything it says for yourself?

being helpful is apparently not the goal with that.

If understanding something requires a certain body of background knowledge, then the most helpful thing you can tell someone who doesn't have that background knowledge is that they need to acquire it if they want to understand that thing. Nature does not care how hard it is to understand something, and ultimately our scientific theories have to accurately predict what Nature does. If the only way our theories can do that is to require a certain body of background knowledge to understand them, then that's the requirement Nature has set, and ignoring it does not help anyone.

 

[PeterDonis]

it should be at least attempted to give a more basic, level appropriate, "pop-sci-like" digested explanation.

What if there is no such explanation? What if every possible "pop-sci-like" attempt at an explanation is wrong, or leads to misunderstanding instead of actual understanding?

 

[TeethWhitener]

And that means you are arguing for pop sci sources to be considered valid references for a PF discussion.

This kind of hardline attitude is a great way to dampen people's enthusiasm for science and discourage curious people from actually studying science. Where should people go with questions about what they read in pop sci? Should we rely on the "go read a real textbook" faction of PF to sort them out?

What if every possible "pop-sci-like" attempt at an explanation is wrong, or leads to misunderstanding instead of actual understanding?

Why limit it to pop-sci? PF has an entire section on physically meaningless quantum interpretations populated by very smart professional scientists.

Honestly, @PeterDonis, in this thread it just feels like you're trying to win an argument rather than consider other viewpoints. And you're very good at arguing, so people are more often than not going to give up getting their point across, even if they're ultimately right.

 

[russ_watters]

This kind of hardline attitude is a great way to dampen people's enthusiasm for science and discourage curious people from actually studying science. Where should people go with questions about what they read in pop sci? Should we rely on the "go read a real textbook" faction of PF to sort them out?

It's worth noting that @PeterDonis 's reply in the subject thread was just "You would be better served..." The thread was not locked or deleted. Nobody was turned away, so they need not necessarily go anywhere else.

Should we be more willing to give half-baked answers to quarter-baked questions? That's a tough one. Most(all?) of the hard science mentors here are professionals/professors and it goes against the job description to do that, so it will be a hard sell. And even if we do, it is still important to make people aware that they aren't getting the real story, but rather a comic-book version of it. Because we also still get a lot of people posting comic book science they need our help to get published. Maybe nobody's ever told them? It's a disservice to not tell people when you/they are glossing-over the "science" part of the science.

 

[hutchphd]

Honestly, @PeterDonis, in this thread it just feels like you're trying to win an argument rather than consider other viewpoints. And you're very good at arguing, so people are more often than not going to give up getting their point across, even if they're ultimately right.

I point out that this is an ad hominem attack and should be recognized as such. I don't think it mean spirited but care should be taken to recognize such arguments as obfuscatory and seldom useful.

 

[AndreasC]

And that means you are arguing for pop sci sources to be considered valid references for a PF discussion

Maybe I'm misunderstanding what "reference" is supposed to mean in that policy, because to me someone asking a question about something they learned from pop sci is completely different to someone using pop sci as a reference for an answer or to make a point.

What if there is no such explanation? What if every possible "pop-sci-like" attempt at an explanation is wrong, or leads to misunderstanding instead of actual understanding?

Then you say "I can't possibly explain this to you at any capacity unless you understand x". But more often than not that is not the case.

 

[Frabjous]

Do we now report threads that do not originate from valid technical sources? Will the mentors close them?

 

[berkeman]

Do we now report threads that do not originate from valid technical sources? Will the mentors close them?

We get reports all the time for unacceptable references in the technical forums. If they are being used as the basis for a technical discussion, then we'll probably do something about it (ask for better references, or maybe close the thread).

 

[Frabjous]

We get reports all the time for unacceptable references in the technical forums. If they are being used as the basis for a technical discussion, then we'll probably do something about it (ask for better references, or maybe close the thread).

Suppose I do not understand something that Hawking said in A Brief History of Time. Given that it is a pop science reference, should a thread on the topic be blocked until I reference one of his technical papers?

 

[PeterDonis]

Suppose I do not understand something that Hawking said in A Brief History of Time. Given that it is a pop science reference, should a thread on the topic be blocked until I reference one of his technical papers?

Short answer: Until you reference a valid source, yes.

Longer answer: Depending on what was said, you might be able to find an easier source than one of his technical papers. Much of that book was not about his personal research on things like Hawking radiation, it was just about our best current model of the universe, which any basic cosmology textbook will cover. For that matter, even the Wikipedia articles on the subject have been used as references for PF discussion, because those particular articles aren't contentious enough to have been the subject of Wikipedia "edit wars", and the articles (at least the ones I've seen referenced on PF) have copious references to textbooks and peer-reviewed papers, so the sources for the content of the articles are clear.

In fact, a big part of the message of PF's policy on valid source is: If you read something in a pop science source, look for valid sources before you ask about it. Very often a quick Internet search will find you a useful reference: a Wikipedia page, or university course notes (there are lots of these online), or something similar.

One note: For books like Hawking's, because of his stature, you might well be able to ask a question about something he said in a Brief History of Time and get an answer on PF before anyone spots that it's a pop science reference. But you will still find that, if you try to follow up with further questions, you will very soon reach the point where someone points out that you're using a pop science source and you can't learn science from pop science sources. That's because that statement is true. It's often an inconvenient truth, but that doesn't make it any less true.

(In some cases, a particularly generous poster might even give you a better reference instead of asking for one. PF does not forbid people from exercising generosity. But you should still be aware that it's generosity.)

 

[KurtLudwig]

I hope PhysicsForums will not shut out current members if they are only at the undergraduate level in physics and mathematics. In a local university library, I attempted to read in Gravitation, by Misner, Thorne and Wheeler. I will never be able to understand the mathematics of general relativity presented in this book, yet I would like to continue to ask questions about gravity. Many members will need physicists and mathematicians of PhysicsForums "to explain the results of these equations" in a verbal format.

 

[PeterDonis]

I hope PhysicsForums will not shut out current members if they are only at the undergraduate level in physics and mathematics.

Undergraduate corresponds to "I" level, so no, we definitely won't do that.

I attempted to read in Gravitation, by Misner, Thorne and Wheeler. I will never be able to understand the mathematics of general relativity presented in this book, yet I would like to continue to ask questions about gravity.

Have you tried Sean Carroll's online lecture notes?

https://www.preposterousuniverse.com/grnotes/

He also has a book published which is basically an expanded version of these. They are probably a much better choice for an "I" level treatment. MTW is a classic, but I would agree that it's way too heavyweight for many; it's an "A" level treatment of the subject.

 

[Tom.G]

What if...

What if???? Straw man maybe?

you're using a pop science source and you can't learn science from pop science sources.

Well, some learn enough to ask questions about something new to them.

Is it time to put this thread out of its misery?

 

[PeterDonis]

Straw man maybe?

If you think so, please make an argument for your apparent belief that there must be a "pop-sci" level explanation for everything in science.

Well, some learn enough to ask questions about something new to them.

My experience as a moderator of this forum has been that the rare cases where a pop science source actually sparks a good question are outweighed by orders of magnitude by the cases where a pop science source spawns a thread whose signal to noise ratio is so low that it has to be shut down. That is one of the main reasons for PF's current policy about valid references.

 

[PeterDonis]

Is it time to put this thread out of its misery?

I don't know. Do you think there is no value in this kind of discussion?

 

[PeterDonis]

PF has an entire section on physically meaningless quantum interpretations populated by very smart professional scientists.

Even in that forum, we still apply the rules about valid sources. There is an extensive literature on quantum interpretations, including both textbooks and peer-reviewed papers, so in that respect the QM interpretations subforum is no different from any other of our science forums. (Similar remarks would apply to the Beyond the Standard Model forum, which is the other subforum where the subject matter goes beyond "mainstream" science to areas that many view as more speculative.)

 

[PeterDonis]

Honestly, @PeterDonis, in this thread it just feels like you're trying to win an argument rather than consider other viewpoints.

What I'm trying to do is explain the rationale for PF's policy about valid references. I did not make up that policy; it's right there in the rules:

• Acceptable Sources:
  Generally, discussion topics should be traceable to standard textbooks or to peer-reviewed scientific literature. Usually, we accept references from journals that are listed in the Thomson/Reuters list (now Clarivate):
  
  https://mjl.clarivate.com/home
  
  Use the search feature to search for journals by words in their titles.
  
  In recent years, there has been an increasing number of "fringe" and Internet-only journals that appear to have lax reviewing standards. We do not generally accept references from such journals. Note that some of these fringe journals are listed in Thomson Reuters. Just because a journal is listed in Thomson Reuters does not mean it is acceptable.
  
  References that appear only on http://www.arxiv.org/ (which is not peer-reviewed) are subject to review by the Mentors. We recognize that in some fields this is the accepted means of professional communication, but in other fields, we prefer to wait until formal publication elsewhere. References that appear only on viXra (http://www.vixra.org) are never allowed.

My question to everyone in this thread who has argued anything along the lines of "we should be able to ask questions based on pop science sources" is: apparently you think the above rules should be changed, since those rules do not include pop science sources as valid references. How and why do you think the rules should be changed?

 

[TeethWhitener]

My question to everyone in this thread who has argued anything along the lines of "we should be able to ask questions based on pop science sources" is: apparently you think the above rules should be changed, since those rules do not include pop science sources as valid references. How and why do you think the rules should be changed?

“Traceable” in the policy seems to be open to at least two different interpretations. Plenty of claims in pop sci are traceable to peer reviewed literature. I therefore see no reason for the rules to change. However, you seem to be interpreting “traceable” in some stricter sense, perhaps with an eye to how faithfully represented that literature is by a particular pop sci work. So maybe you want the rules to change to reflect that.

 

[PeterDonis]

Plenty of claims in pop sci are traceable to peer reviewed literature.

Yes, and in such cases, our response is generally to try to redirect the discussion to be based on the peer reviewed literature that the claim is traceable to, at least if the discussion goes beyond a simple question and answer or two. (This assumes that the pop science source reasonably represents the literature; I'll address the case where it doesn't below.)

However, many pop science sources cite no literature whatever. This is particularly disappointing with articles that are specifically discussing some new experiment or research--but without any link to the paper that published the research. In such cases, unfortunately, the response is going to be to ask the OP to find a valid reference.

you seem to be interpreting “traceable” in some stricter sense

No. I'm just pointing out that the traceable cases are rare in comparison to the total number of questions we see based on pop science sources. And in my experience edge cases where the boundary of "traceable" is not reasonably clear are extremely rare.

perhaps with an eye to how faithfully represented that literature is by a particular pop sci work

If we can trace the actual literature, cases where the pop science article misrepresents the literature (which are unfortunately common, even when the pop science article is written by a scientist who should know better) are fairly easy to handle, at least as far as pointing out the discrepancy goes. (The OP of such threads doesn't always receive the news very well, but that's a separate issue.)

 

[AndreasC]

Something can be traceable even if the op doesn't or can't trace it though.

 

[Rive]

How and why do you think the rules should be changed?

I think the source of the source problem is, that the acceptable sources list just does not really fit the Curious Average Joe. Who will 'learn' some popsci first, and then can only expect to get a wall in the face here for the first question.

Not really a good hook for new member retention.

As I see it, while references, sources and starting points should be clearly distinguished by requirements (and regarding references, what we have here is just perfect!), some kind of recognition should be granted for valid questions originating from some (!) popsci sources as starting point (preferably just as a starting point: possibly 'at Mentor discretion' as this approach already exists for other fuzzy cases).

With making it very clear, that popsci questions will result in popsci answers of limited value only, and unless the topic can steer away from the starting point it'll be short lived.
But still, officially recognized, so no one should feel shame for asking.

That's how I think.


So instead of a 'we don't do that' it should be some kind of 'you should move away from that' policy, which - as far as I see it - is actually what's often happening, just unofficially.

 

[vanhees71]

I think it's a kind of a dilemma. On the one hand I like @PeterDonis pretty strict policy on admitting only "papers in refereed journals" and textbooks as valid references in PF. Experience shows that the former are indeed the most reliable sources of valid results of research. Of course, even for them there's no guarantee that there are mistakes in them too since we all are more or less carefull reviewers ;-)). Textbooks are a little less reliable (at least newer ones since nowadays no scientific publisher can afford are true lectorate, but that's another story). In contrast popular-science books or even some webpage or youtube video are usually very unreliable. The reason is that it is utmost difficult to write a good popular-science text without to some extent being unprecise. That's because the only reliable language to express science accurately is the use of mathematics, which you can't do when writing a popular-science text.

On the other hand, popular-science texts are important to inform the interested public about current research. After all much money paying our experiments and the theorists trying to understand them in fundamental research is tax payers' money, and thus the public has a right to be informed about the results. Outreach, nowadays, is thus more and more important also for the funding agencies, and in any proposal for funding we have to include a section about possible outreach activities.

So it's also not a good idea to ban all popular-science texts from discussion in physicsforums, and one should not be too quick with cutting of a discussion, only because the OP cannot deliver a "valid reference". Of course, one can and should point out possible inaccuracies in popular-science texts and also give references to the scientific literature in the discussion.

Last but not least there are also quite reliable (semi-)popular sources like Wikipedia, which is amazingly good (the English version usually somewhat better than the German; other languages I'm unable to judge).

Finally, I don't think we need a change of the general PF rules on "acceptable sources". I only think one should not completely ban discussion of popular-science sources, even if an interested lay man cannot point to a scientific reference to its subject.

 

[TeethWhitener]

I'm just pointing out that the traceable cases are rare in comparison to the total number of questions we see based on pop science sources.

@vanhees71 , @Rive , and @AndreasC above have given pretty good synopses of how I feel here, but I think this quote above is where you and I likely disagree the most. Maybe it’s because we frequent different sections of PF, but I don’t often come across pop sci discussion that can’t be traced fairly straightforwardly to the literature, even if that means I have to do some googling. (I’m leaving aside cases where OP says some variant of “I read somewhere,” where I think asking for concrete sources is acceptable/necessary.)

And yes, the onus isn’t on the responder to find the literature on which a pop sci article is based, but frankly the onus isn’t on the responder to respond at all. If the OP is going to get hit with “go get a real string theory textbook” every time Michio Kaku opens his mouth, that OP is likely not to come back to PF for answers, which makes sense because they didn’t get an answer from PF. They got told to go away.

 

[pinball1970]

I went back to the beginning of the thread, the comment was,
"This is a pop science book. You would be better served by reading actual textbooks or peer-reviewed papers."

I don't see what is wrong with that. It is up to the poster to think about what that means and what to do next.
Realising that popsci does not teach any physics, even though the book is authored by a well known physicist, is not that obvious to a non physicist. (From personal experience)

I do not see the problem explaining that, then directing them where appropriate?
"A good beginners book is ...(insert beginners book here)."
I have seen that lots of times in different forums EDIT: On PF.

 

[vanhees71]

Per se, there's nothing to say against the content of the 1st paragraph, but don't you think it's repelling interested laymen from posting again and/or following the discussion further? Compare this to your own formulation in the last paragraph, which is the much better way of expression the same thing!

 

[Vanadium 50]

I think "what if" is not a very productive direction - we're not going to come up with a flowchart that addresses every possible case. Further, trying to set policy by looking prinarily at edge cases usually just makes a mess.

If someone says "I heard that", it is entirely appropriate for PF - not just the staff - to ask "where?".

If the answer to that is not a very good source, PF. - not just the staff - can say so. "This is a popularization and cuts a lot of corner"s should be an acceptable answer. And yes, I know not everyone accepts it.

On the general issue of popularrizations, if they were as good as textbooks, there wouldn't be any textbooks, Who would bother?

Finally, if you want to argue that threads should be closed by non-participating mentors, because having participating mentors close it is n not a good look ("Shut up, he explained"), I agree. But that's hardly worth a thread on it.

 

[phinds]

My two cents: changing how we deal with pop-sci and references in general would be a TERRIBLE idea. Why? Very simple. This forum is enormously successful for one simple reason. The moderators.

Force the moderators to deal with a lot more crap than they already have to deal with and you start driving THEM away and thus would begin the downfall of this forum as such an excellent site for science.

Also, just my opinion, I don't find the moderators to be overly harsh on ignorant newbies. To the contrary, I find it astounding just how patient they often are.

 

[vanhees71]

Yes, I'd also not change the general rules, which are successfully applied for years. One should only apply them with sufficient flexibility not to deterr new posters by a very harsh reaction to any pop-sci reference (or even on "I've read somewhere, that...").

 

[AndreasC]

I think "what if" is not a very productive direction - we're not going to come up with a flowchart that addresses every possible case. Further, trying to set policy nu looking prinarily at edge cases usually just makes a mess.

If someone says "I heard that", it is entirely appropriate for PF - not just the staff - to ask "where?".

If the answer to that is not a very good source, PF. - not just the staff - can say so. "This is a popularization and cuts a lot of corners
should be an acceptable answer. And yes, I know not everyone accepts it.

On the general issue of popularrizations, if they were as good as textbooks, there wouldn't be any textbooks, Who would bother?

Finally, if you want to argue that threads should be closed by non-participating mentors, because having participating mentors close it is n not a good look ("Shut up, he ecxplained", I agree. But that's hardly worth a thread on it.

I don't see anything wrong with what you are saying here. I don't object with telling people that pop sci is not a good source, etc. But, after that is done, then what I believe is that if the question nevertheless makes sense but op is not advanced enough to understand a highly technical discussion, it is preferable to attempt a level appropriate answer to the extent it is possible, or say that you have no way to answer it in a level appropriate manner and point to the required background, rather than closing the thread or telling them they should read graduate level original papers to have their question answered (which won't happen).

Now, if the question does not make any sense, it's still instructive to tell someone their question doesn't make sense and why.

Or you can not answer at all. Nobody is forced to answer obviously. I just think that telling people to basically come back when they finish their Masters can be kind of frustrating and off putting to some people.

After all, the general public funds research, and some highly theoretical disciplines don't "pay back" very much. They are worthwhile because they broaden human understanding. What good is that however if none of that "trickles down" to anyone but highly specialized experts? And what is a better place for the general public to come in contact with legitimate scientists and ask them questions?

For what it's worth, I don't think changing the general rules is necessary either, I agree with what @vanhees71 said.