Invisible science and how it’s too easy to fool yourself

If you want the full effect of this post, watch the video below first before reading further.

[scroll down for the post]

.

.

.

.

.

.

.

.

.

.

So, did you see it? I did, probably because I was primed to watch for it, but apparently 50% of subjects don’t!

I heard about a really interesting psychology experiment today (and I LOVE these kinds of things that show us how we’re not as smart as we think we are) called the invisible gorilla experiment. The set up is simple, the subjects watch a video of kids passing balls back and forth. The kids are wearing either red or white shirts. The object is to count the number of times a ball is passed between kids with white shirts. It takes concentration since the kids are moving and mixing and tossing fast. At some point a gorilla walks into view, beats its chest, and walks off. Subjects are then asked if they saw a gorilla. Surprisingly (or not- because it’s one of THESE kinds of experiments) 50% of the subjects don’t remember seeing a gorilla. What they’ve been told to look for and pay attention to is the ball and the color of shirts- gorillas don’t figure in to that equation and your brain, which is very good at filtering out irrelevant information, filters this out.

Anyway, it got me thinking about how we do science. Some of the most interesting, useful, exciting, groundbreaking results in science arise from the unexpected result. You’ve set up your experiment perfectly, you execute it perfectly, and it turns out WRONG! It doesn’t fit with your hypothesis, but in some weird way. Repeat the experiment a few times. If that doesn’t fix the problem then work on changing the experiment until you get rid of that pesky weird result. Ahhhh, there, you’ve ‘fixed’ it- now things will fit with what you expected in the first place.

Most of the time spurious, weird results are probably just that- not very interesting. However, there are probably a lot of times when there are weird results that we as scientists don’t even see. We don’t expect to see them, so we don’t see them. And those could be incredibly interesting. I can see this as being the case in what I do a lot, analysis of high-throughput data (lots of measurements for lots of components at the same time- like microarray expression data). It’s sometimes like trying to count the number of times the kids wearing white shirts pass the ball back and forth- but where there are 300 shirt colors and 2500 kids. Ouch. A gorilla wandering into that mess would be about as obvious as Waldo in a multi-colored referees’ convention. That is, not so much. I wonder how many interesting things are missed and how important that is. In high throughput data analysis often times the goal is to focus on what’s important and ignore the rest- but if the rest is telling an important and dominant story we’re really missing the boat.

I’ve found that one of the best things I can do in my science is to be my own reviewer, my own critic, and my own skeptic. If some result turns out exceptionally well I don’t believe it. Actually there’s an inverse correlation between my belief and the quality of the result with what I expect. I figure if I don’t do this someone down the line will- and it will come back to me. I try to eliminate all the other possibilities of what could be going on (using the scientific method algorithm I’ve previously described). I try to rigorously oppose all my findings until I myself am convinced. However, studies like the invisible gorilla really make me wonder how good I am at seeing things that I’m not specifically looking for.

 

3 thoughts on “Invisible science and how it’s too easy to fool yourself

  1. Pingback: Cool example of invisible science | The Mad Scientist Confectioner's Club

  2. Pingback: Fact from fiction: The scientific method is alive and well | The Mad Scientist Confectioner's Club

  3. Pingback: Eight red flags in bioinformatics analyses | The Mad Scientist Confectioner's Club

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>