Peer Review

Q: What’s the fastest way to piss me off?

A: Reject peer review outright

 

Let’s talk for a minute about peer review, and address some of the phenomenally dumb things I’ve seen posted on various forums about the subject. For example, this lovely little snippet from a forum I used to frequent:

 

For near 4 and half decades the AMA has been criticized for its actions. It is a corporate entity that has warmed its way into being a policy maker for the US government. Its a politically entity and the crap its prints in these stupid papers are as worth as much as used toilet paper.

You are not scholarly, educated or well read for parroting this crap…your just gullible.

 

Yep, that's the brain all right...

Lovely, lovely person.

 

What a lovely person! Of course, he’s completely wrong in just about every meaningful way, but I’m going to focus on just one thing here – that the papers published in JAMA (the journal of the American Medical Association) are worth as much as used toilet paper.

 

This is a particular attitude I find disturbingly common among people who are completely clueless about science – the idea that peer-review is worthless, or corrupt, or <insert alternative adjective meant to explain why extensive study by trained professionals analyzed for errors by other trained professionals is somehow equal or even inferior to the uneducated opinion of a backwards yokel who probably flunked high school science>. It’s almost always used in an attempt to discredit evidence showing why they’re wrong.

 

So why is this argument dumb? Let’s just assume the person we’re talking to doesn’t know the first thing about how science works today. (A fair assumption – if they did, they wouldn’t say anything quite so stupid). Well, who’s the target audience of peer-reviewed journals? Is it the layperson? Well, an individual article from Nature costs between $18 and $32; a year’s subscription to JAMA costs over $200. So it’s not exactly the kind of thing the average Joe would read, especially given that the “average Joe” would get about as much for it as he’d get reading the manufacturer’s handbook to an F-16 bomber.

 

No, the people who read JAMA are primarily universities, medical students, medical scientists, and doctors. They use the information therein to inform themselves and their pupils. This is the basic mechanism of propagation of information in science. Your doctor? He learned what he learned largely based on the information in that “toilet paper”. Not only that, but medical scientists creating new advances cite articles from JAMA very regularly, to the point where it is one of the most-cited medical journals in the world. Which in turn implies that people making the breakthroughs want to publish there, because more people will see it and more people are likely to know about their research. The experts are all pretty much in agreement – this is a good source. I somehow doubt you know more than they do.

 

But let’s ignore them for a moment and just focus on your doctor. with all that in mind, do you go to the doctor when you have an itchy rash on your genitals, or is his opinion worthless? After all, his training is largely based off of the things he picked up from those “worthless scraps of toilet paper”.

 

“Well, now that I know that he gets his information from those corporate shills, I’m never seeing him again!”

 

 

…Oh dear. Well, on the bright side, natural selection is still a thing. It’s just a shame it doesn’t really work like that.