Judge gets scientific method better than scientists

[anorlunda]

https://reason.com/wp-content/uploads/2021/07/b49101a8-ad23-44dc-9500-ecec551c08b8.pdf

This was unexpected. In a 55 page decision from the Superior Court of DC, the judge said the following.

[A]s gatekeeper, the Court cannot allow an expert to testify concerning documents and articles that they have reviewed, unless the expert can establish that they have used some technique or methodology that systematically gathers, organizes and catalogs the documents such that another expert with similar training could follow the same procedure and arrive at the same result...
The methodologies of the expert must be grounded in the scientific method, such that another person with similar expertise could replicate them. See Daubert 509 U.S. at 591. Reviewing a selection of documents, summarizing them, and giving an opinion about their conclusions is not a proper methodology grounded in the scientific method

The scientist litigator, and his expert witnesses said:

When asked about the methodologies that she used in this case, Dr. Oreskes responded: "If you want me to tell you what my method is, it's reading and thinking. We read. We read documents. And we think about them."

It seems to me that the judge gets it, and the scientist(s) did not.

This example could also serve to illustrate the difference between a post in a PF thread and a paper submitted to a peer reviewed journal. Here on PF we strive to educate about science and engineering as known in the peer reviewed literature. If some of our members do new science, they prepare it for the higher standard of peer review, not post it here. The judge did a good job of explaining the difference.

 

[Andrew Mason]

I agree that the judge was correct.

You would be amazed, though, at what kind of bogus opinion evidence gets admitted in courts.

For decades, the FBI used comparative bullet lead analysis as evidence that a bullet used in a crime came from the same box of bullets found in the possession of the accused. This began in the 1960's and continued until about 2005 after the NAS declared it to be unreliable. Forensic Analysis Weighing Bullet Lead Evidence. ,

The FBI had to discontinue use of the evidence and thousands of convictions had to be vacated.

Other bogus 'science' includes 'shaken baby syndrome' which has never been established as valid, fire marshalls providing expert opinion as to the cause of fire (the case of Cameron Todd Willingham will make you cringe) and forensic "bite mark" evidence.

Although this judge got it right, many don't. The problem is that once a new 'science' gets accepted by a few courts it takes off and is difficult to stop.

AM

 

[Bandersnatch]

The scientist litigator, and his expert witnesses said:

It should be noted that Dr. Oreskes is a historian, and what she described looks representative of a historian's modus operandi. They literally read documents and think about them. They don't run experiments, they don't take measurements. There's no results to replicate. Their methods consist of digging up sources, critiquing their trustworthiness, and arguing for a coherent narrative on their basis.
It looks to me like in the judge's opinion no historian can ever provide expert testimony, since the jury should just read the relevant documents themselves.
Which... I don't know. I can see some merit to that, since in the court of law one should strive for maximal objectivity, and history can be very subjective. But on the other hand, this outright dismisses a vast body of knowledge as irrelevant.

 

[anorlunda]

But on the other hand, this outright dismisses a vast body of knowledge as irrelevant.

Here, the judge describes in more detail the criteria for expert witnesses. Do you think that is inadequate?

In determining whether an expert opinion is based on specialized knowledge and has used reliable methodologies, the Court will look to such factors as: (1) “whether the theory or technique … can be (and has been) tested;” (2) “whether it has been subjected to peer review and publication;” (3) “the known or potential rate of error;” (4) “the existence and maintenance of standards controlling the technique’s operation;” and (5) “whether the technique has been accepted by the scientific community.” Daubert, 509 U.S. at 593-94. Determining reliability is a flexible inquiry that focuses “solely on principles and methodology, not on the conclusions that they generate.”

I have never seen historians reviewing historical documents be able to give error bars on their conclusions. But maybe I'm wrong.

 

[Bandersnatch]

Do you think that is inadequate?

No, I think the opposite. I think that it is holding history to standards only empirical sciences can match, therefore excluding historians from ever being able to give expert testimony. It's basically saying the professional expertise of a historian is inadmissible, and can be equally replaced by letting the jury just read the sources themselves.

I have never seen historians reviewing historical documents be able to give error bars on their conclusions.

Yes, that's the point.

 

[Choppy]

Yes, but really I think we all know that the best way to establish the expertise of a witness...

Correct timing of a 1955 Bel Air Chevrolet with a 327 ci engine and a four barrel carburetor

 

[sbrothy]

As a European it's refreshing to see at least some degree of sanity in the US legal system.

Karl Popper would've been proud had he been alive.

EDIT: Or maybe "proud" isn't the right word. At least he accomplished something of lasting value.