Interpreting Medical Data: No, Kelly McGonigal, It Didn’t “Work For You”


So… Boy, that name is a double misnomer. Surely it’s an information website! And surely it’s all about preventing disease! I bet I’ll find some great information on vaccines there!

…Oh. Shit. Because of course it isn’t. That would be too much of a positive thing. Instead, PreventDisease is your typical alt-med conspiracy farm. The kind of people who will, with a straight face, say “EVERYTHING IS A LIE!!!” (I’m not even kidding!) and then go on to recommend homeopathic treatment for cancer. Well, I can say with confidence that they’re at least partially right – everything on their website is a lie. Okay, maybe not everything, but it’s fair to group it in with Mercola, NaturalNews, and their ilk – if a medical treatment is proven to work, they’re bound to reject it, and if it’s proven not to work and falls under the purview of woo, they’re going to accept it. That incredibly bogus list I linked above (the one linked over “shit”)? Just the tip of the iceberg.

That execrable list is not my target today (if you want to see it taken down, mosey on down to Science-Based Medicine). No, I picked up on something which piqued my interest on their main page. A video on their youtube channel, “PreventDiseaseTV“, by someone who appears to be a real scientist (from Stanford, no less). Kelly McGonigal, a health psychologist. By all means, her credentials seem to stand up on their own. Unfortunately, her video… Well, it doesn’t.

The first part really isn’t that bad – it talks about not relying on single studies, but rather on an understanding of the whole field of study. I found it hilarious that PreventDisease kept this part in the video, for the simple reason that this is something they never fucking do. Like, take the discussion of the homeopathy cancer paper above – analysis of the whole field of study says, in no uncertain terms, “homeopathy works no better than placebo for all examined treatments, and this is because it’s literally JUST WATER”. Or what about vaccines? The first question in their “stumpers” involves evidence that vaccines are effective. It took Steve Novella’s 12-year-old all of 22 seconds to find a study showing exactly that. Well, 400 studies to be more accurate. PreventDisease, just like various other sites in the same vein, never examines the whole field of research. If it did, it wouldn’t post tripe like what it does. It wouldn’t claim that sunscreen is highly toxic, or that vaccines don’t prevent disease. Or, for that matter, that Naturopathy holds answers to modern medical problems (or, let’s be perfectly honest here, ancient ones).

Wanna know the fun thing about medicine that doesn’t work? Me too.

But her comments on how to interpret studies is just… really, really bad. Look, when a study finds no statistically significant evidence that something works, it usually means that that something doesn’t work. If there is a tiny minority that shows a positive result, the question is, “why?”. If you give 50,000 people your medicine and 10 get better, is the most rational response “it worked for them” regardless of other factors? What if you give 50,000 people your medicine and 50,000 a placebo, and 12 given your medicine get healthy and 10 given placebo get healthy? Is it likely that it worked for those 12 people? Or maybe just 2 of them?

Look, while this is a complex subject, the basic picture is not hard to understand, and the basic picture alone is enough to point out just how wrong McGonigal is. For example, in a study on homeopathy, you might have a handful of people who do appear to get better after taking homeopathy. It’s not significantly different from placebo, though – these people didn’t “benefit significantly” from the treatment; it’s almost certainly regression to the mean, the placebo effect, or any number of other confounding effects that real medical scientists know how to look out for.

The data that your own body and brain gives you is nice, but it’s just a sample size of 1. If I have the flu, take Oscillococcinum, and feel better, that does not mean that “oscillococcinum worked for me” – because oscillococcinum doesn’t work better than a placebo, because in order for it to work you’d have to throw out half of physics and chemistry. By trusting your own personal experience over scientific research, you open yourself up to a colossal number of biases that (hopefully) will not be present in the research. It’s just as I said – your personal experience is nice, but N=1. The fact that a health care professional wouldn’t understand this is… disappointing. This is terrible and dangerous medical advice. Treatments either work under certain conditions or they do not. “It worked for me” is not a valid response when the treatment in question has been shown in trials not to work to any significant degree beyond placebo.

Leave a Comment