Personal Experience: N = 1

What is it with personal experience? I hear this time and time again from people (usually older than me to a degree where I’m just “some young whippersnapper”) – “Well you have your facts and figures and published peer-reviewed papers and scientific authorities and fundamental understanding of the subject, but you’re still wrong because my personal experience says so!”

“Sure, it may be geometrically impossible, but my cousin swears he made one in wood shop the other day!”

The most common topics, of course, being things to do with medicine that just ain’t so. For example, I had a riveting conversation with an in-law the other day about whether or not the flu vaccine gave one of her friends the flu. (Pro tip: it really didn’t.) When I pointed out that no, the flu vaccine does not give you the flu, she quite literally said “you have your facts and figures, but I know real people who this happened to”. Guys, pro tip. “Something which is scientifically impossible happened to my friend” is not a good argument.

Personal experience is a cruel mistress. On one hand, experimentation and observation is the basis of all science. After all, if we cannot observe a phenomenon, we cannot reasonably claim that it exists. Without the observations of gravity performed in the 1600s, it would not have been reasonable to assume that masses attract each other. Without the observations of germs performed by Louis Pasteur, we wouldn’t have good reason to assert that we knew the cause of things like influenza. But at the same time, human observation is innately error-prone, and there are a lot of ways we can go wrong in our observations. A lot of ways.

EVEN THE FAMILY CIRCUS GETS IT.

If you take something labeled as a remedy for some ailment, and then the ailment goes away, we’re basically primed to associate the remedy with the resolution, regardless of whether or not it actually had anything to do with it. In fact, you can prime pretty much any such association – tell someone that the flu shot can make them sick, and if they get the flu soon afterwards, they’re going to blame the shot… even though that simply isn’t possible. Or how about homeopathy – I have friends who swear that what amounts to a small dose of water and maybe a sugar pill cured them of their ills.

Personal experience is meaningful, but has very serious limitations. In theory, it’s the ultimate arbiter – after all, why trust those things written in a book somewhere by people you’re told are reliable over what your own senses tell you? But in practice, humans run into a ton of biases and errors in their observations. The sheer number of things that can go wrong, from simple biases and mistakes to straight-up hallucination, is really staggering. This is part of why medical publishing works the way it does. Small sample sizes lead to biases and outliers being extremely influential in the data. And individual experience? Your own, personal experience? Well, that’s a sample size of one. And as unbiased as you may be, any personal experience that would lead you to question scientific consensus should be taken with a massive grain of salt. Whenever such an experience comes along, you need to check yourself – make sure that you interpreted what happened correctly; make sure that there wasn’t some other explanation; make sure that you didn’t draw a causative link between two things whose sole connection was “they happened close to each other”.

If children can understand it, so can you!

This is why scientific papers are so important. In a large-scale clinical trial, you can eliminate most biases. You can attempt to establish as strong of a correlation as possible. You don’t rely on the flimsy correlations so often present in personal experience, where it may well seem that homeopathy cleared up your cold (it went away on its own) or ozone therapy kept Keith Richards alive (he’s just the luckiest fucker on the face of the planet) or a jab gave you the flu (you got the jab during peak flu season and it takes a week or two to set in) – rather, you examine large datasets to determine whether or not this was a fluke – whether there was anything to the correlation. This is why, when my personal experience on medical matters contradicts the available well-designed peer-reviewed research and I am not completely sure that I could not be wrong – if there is any way I could have reasonably been mistaken, and the papers are not clearly flawed – I defer judgment. I feel comfortable in doing this primarily because of the fallacies I’ve seen – my father taking hydrogen peroxide because he thinks it makes him feel better (when in reality it simply has no positive effect); my friends refusing the flu vaccine because it made them sick (when in reality you can’t fucking get the flu from the flu vaccine); et cetera, et cetera, et cetera.

Personal experience is nice. But treating it as the end-all, be-all, when it’s so open to biases and errors… Not a good plan. Not a good plan at all.

Leave a Comment