Lies, Doggone Lies, and Statistics
Columns, Featured, Spirit in the Material World — By Stephen Simpson on November 30, 2009 at 1:00 am
My wife TiVo’s The View and watches it at night. Sometimes I’ll catch a snippet while I’m waiting for my turn to watch Friday Night Lights, The Office, or Sportscenter. I do not watch The View on purpose. Ever. When I do it’s only for the sake of my marriage. Let it go.
Anyway, I heard one of the women on The View, I think it was the redhead, make the following statement: “You know, children who were molested are more likely to become pedophile’s as adults.”
“Stop it!” I snapped. My wife, without looking up, said, “I know, I know, but it’s The View. They’re not going to bother with details.”
Ladies and gentlemen, when it comes to statistics, especially the kind linking victimization as a child to perpetration as an adult, details are everything.
Here’s what Miz Coffee Talk didn’t tell you: The overwhelming majority of adults molested as children are not pedophiles. They are only slightly more likely than members of the general population to molest children. The way that she said it might lead you to believe the opposite, that victims of molestation will probably grow up to sexually abuse children. Presenting information in this half-assed manner leaves out the humbling reality that most victims manage to work through the trauma and healthy adult relationships.
Here’s another example of shoddy scientific reporting. A study out of the University of Louisville said that swingers – married folks with an open sexual relationship – have better marriages than those prudish “forsaking all others” types. I’ve heard of husbands using this study to talk their wives into a little extra-marital promiscuity. But the research indicating that “swinging” promotes healthy relationships has a pretty big hole in it: researchers interviewed only active swingers. This is the equivalent of interviewing everyone at a bowling alley and discovering that most of them enjoy bowling. I have found no studies about people who stopped swinging, and those are the folks that would give us the real dirt.
Being committed to the truth means more than memorizing Scripture. It means understanding at least a little bit about scientific method. You’ve probably noticed that postmodernism is over and it’s okay to call people stupid again. Christopher Hitchens and his ilk make the claim that it’s preposterous to rely on faith in the unseen when we have science to tell us about truth. Let’s forget for a second that science makes huge philosophical assumptions about epistemology. Let’s overlook its miserable track record when it comes to truth. We’re still left with frequently sloppy scientific method and, even when the methods are sound, misleading use of the data.
The academy considers a correlation of r= .6 a strong statistical relationship. Put simply, a study that finds variable A relates to variable B only thirty-six percent of the time is considered a scientific breakthrough. In the realm of science, that’s a very strong relationship. The remaining sixty-four percent of variance, however, gets little attention. When we find even a thin relationship between things like breast-feeding and intelligence, diet soda and weight gain, sexual orientation and brain structure, or cohabitation and divorce, we shout it from the rooftops as a scientific breakthrough. When the media gets hold of the information, they report it as truth. Scientists, much less reporters, seldom remind the public that there’s a lot that we don’t understand. In fact, we don’t understand most of it.
Christopher Hitchens
We need more amateur Christian scientists. I do not mean creationists or people steeped in the data of the Family Research Council. We need people who understand the scientific method. Science is not as objective and reliable as Chris Hitchens implies. We need more folks who can spot bad science and call foul when it’s used to mislead people.
This is not an anti-science rant; it’s a rant against bad science. People don’t associate good science with Christians. Hopefully, our integrity and commitment to the truth can make us some of the best scientists.
10 Comments
Too many people out there think science is somehow exempt from basic philosophy and thus disregard the lessons of Ecclesiates, Proverbs and Socrates. The only sane, fundamental assumption to any logical and/or scientific exercise is that we are ignorant and biased. Here are some helpful rules when approaching science:
1) A scientist is always biased in choosing what to study and how to study it. It is not possible to be unbiased, anyone who claims otherwise is a fool.
2) Everyone observing results of a study is biased in interpreting it, including (and often especially) the expert scientist in charge.
3) No amount of correlation EVER equals causation. Period. No exceptions. So long as you only have correlative evidence, your theory is unproven and alternate explanations are possible. Seriously, stop trying to think of exceptions because there are NONE.
4) Causation is really, really damn hard to prove. So it’s ok to use strong correlations with a plausible theory of causation sometimes. Such theories can be useful models even if we’re not actually sure they’re correct. Just don’t ever forget that the theory could be totally wrong regardless of how useful it is.
5) Having an experiment be “repeatable” is no substitute for actually having it be repeated.
6) Statistics are not data that results from a study. Statistics are always a biased interpretation of data that results from biased study (see rules 1 and 2). Further, statistics are usually among the most biased interpretations that can be derived from the data resulting from a study as they usually are far too succinct to be reliable.
That said, the scientific method is one of the most useful things we have out there for technological and medical advancement. It’s like democracy, a terrible form of government, but still the best one available to us.
This is why it is not enough to do a single, simple study and declare you’ve found truth.
Well said, Stephen. You can rant to me anytime.
Unless its about the way I wear my socks.
Rob,
There is a very strong correlation between the way you wear your socks and my inability to make it through life without huffing glue.
Luv,
Steve
Thanks, Stephen, for a good rant.
I’d like to add that while this post is a “rant against bad science,” it is also a rant against poor critical thinking skills.
We should work toward a better understanding and application of scientific method, sure. But let’s apply the kind of thinking that prompts good questions to all areas of life.
I completely agree with you, Christina! This is an issue of applied critical thinking. (Which you, Stephen, have done really well here.) Thanks!
I clicked on this article solely on the splash image from the View.
i thought it was a funny picture.
glad i clicked it.
Thanks for your example. It’s pretty standard “best practice” for pastors to ask applicants who want to work with children if they had been the victim of sexual abuse as a child. Insurance companies suggest this question to churches and churches comply, wanting to have good insurance rates and wanting to protect children.
I can’t ask that question though. Applying to work with children shouldn’t strip you of your dignity. We do reference checks, national sexually offenders database searches, county criminal records searches, applications, and interviews.
But I worked with enough abused kids as a mental health caseworker to not have the heart to subject them to a frivolous and hurtful line of questioning like this.
A great journalist/scientist calling out these faults is Ben Goldacre for The Guardian (a British news paper).
I concur, the problem is that loose connections are missunderstood by the news media who proclaim “truth” where even the original scientist wouldn’t, which then gets claimed by talk-show hosts to show how smart they are. But until we (the general public) call news media to account for this, nothing is going to change.
Statistics can be used to prove anything you want. 76% of people know that.
And anything can be claimed as truth, even bold-faced lies – just look at the Obama birthcitificate debacle.
Great rant! Love it!
I am especially excited/challenged/frustrated with the last two sentences. It is difficult to hear and discern those voices who are unafraid to rigorously utilize the tool of science in trying to understand the universe who don’t make it a crusade against some “humanistic” theory.
Anyway, thanks for the article.
36%? seriously? it seems to me that if a relates to b less than half the time, that’s a pretty good sign that a and b aren’t really related. i would think it would need to be at least 75% to declare any sort of correspondence, and even that would be a bit shaky.