The scientific method (as imagined by a biologist-turned-computational-biologist)

“I have approximate answers, and possible beliefs, and different degrees of certainty about different things, but I’m not absolutely sure of anything”- Richard Feynman

“In science, ‘fact’ can only mean ‘confirmed to such a degree that it would be perverse to withhold provisional assent.'” – Stephen Jay Gould

“Computational biology is just biology”- me.

So the genesis of this post is that there are some ‘real biologists’ or other ‘real scientists’ who believe that computational biology is sitting around making things up on computers willy-nilly, or randomly wandering through data to come to some conclusion, or writing code to solve problems that no one (no biologist) actually cares about, or some other such endeavor that doesn’t count as science. In many cases this is simply a field-based cultural perception that they don’t really give a thought to (i.e. they aren’t trying to dis us actively) but others that think a lot about these kinds of issues seem to campaign against us. Of course, there are many examples of computational analysis of biological data that do fall into the categories of non-science, just as there are many scientists who do experiments without appropriate attention to the scientific method and thus fall into the same boat.

Generally, what is the scientific method?

First, here are the steps I think of in the scientific method:

1. Hypothesis formation: in which you determine what the question is that you want to ask. This is generally developed from previous experiments (see step 5 below), but can also be from ‘exploratory’ experiments and/or analyses. Generally, ‘exploratory’ simply means that the hypothesis being tested in the experiment is very broad.

2. Formulation of experiment: in which you figure out how you will ask the question. In science this is to falsify your hypothesis- that is, given this hypothesis how would I show that it ISN’T true. This ALWAYS includes controls- that is, portions of the experiment that should turn out negative and/or positive by your evaluation method.

3. Execution of experiment: do the experiment.

4. Evaluation of initial hypothesis: examine the results carefully. Compare the experimental results against the results from your control experiment. Did the control experiments work? If not, something is wrong with your experiment. If they did, do the results of your experiment differ significantly from the controls? Are the results clear enough to either falsify your hypothesis (generally considered a negative result) or not falsify your hypothesis? No hypothesis is ever ‘proven’, only proven false or not disproven.

5. Formulation of further experiments. If your hypothesis still survives then how else might you falsify the hypothesis. Generally there should be multiple ways that you can think of to falsify your hypothesis- that is, multiple reasons for the results of your experiment that are not the idea put forth by your hypothesis. You have to test these too to try to falsify your hypothesis. If you don’t, reviewers will ask for you to do it- or at least they should. You should not rule a hypothesis ‘supported’ until you have investigated, and discarded several alternative pathways that might have generated the results you’ve seen. So these further experiments may be inverted from your original hypothesis in that you are proposing a new hypothesis that your result is due to something else, then trying to falsify this alternative. Go back to step 2.

6. Reformulation/modification of hypothesis anew. Alternative 1: Your hypothesis has survived! Congratulations, you move to the next step- given that your hypothesis is true (note that you actually have not SHOWN that your hypothesis is true, only that it is not false by a limited number of alternative experiments [see step 5]- which is why I say “given that your hypothesis is true”, not “since your hypothesis is true”), what is the next question that is suggested? Are there implications? Have you seen anything else in the results that seems funny or interesting or hard to explain? Maybe that can be interpreted in the context of a new hypothesis and you can discover new things! Go to step 1. Alternative 2: Your hypothesis was falsified. Too bad, that’s science. Is the original phenomenon or observation you were trying to investigate still interesting? Can you think of an alternative hypothesis that is testable given your experimental and/or data limitations? The phenomenon has to be happening for some reason and you can determine it with enough hard work so get back at it. Go to step 1.

A toy example of how the scientific method works

You’ve found a box. It’s small enough to pick up and sealed closed. Your hypothesis is that the box is completely empty.

Round 1:

  • Hypothesis: there’s nothing in the box
  • Experiment: shake the box
  • Control: shake a box you know is empty
  • Results: there is no sound and you don’t hear anything rattling around just like the positive control box
  • Conclusion: the hypothesis is supported, there is nothing in the …. wait, wait, wait, wait! What if there’s something in the box that doesn’t rattle?
Round 2:
  • Hypothesis: there’s nothing in the box (but now you’ve eliminated the possibility that it rattles)
  • Experiment: weigh the box
  • Control: weigh the empty box
  • Results: you can detect no difference in weight between your ‘experimental’ box and your control box
  • Conclusion: the hypothesis is supported, there is nothing in the….. hmmmmmm….. OK. What if what’s in the box doesn’t weigh much at all. Actually what if it weighs little enough to fall in the margin of error for the scale (or other method) you’re using to compare the two?
Round 3:
  •  Hypothesis: there’s nothing in the box (but now if there is something there it doesn’t rattle and doesn’t weigh much)
  • Experiment: take a shotgun and shoot the box
  • Control: shoot an empty box (OK- these are simple controls, but you get the idea)
  • Results: you see that significantly fewer holes in the exit side of the ‘experimental’ box relative to the control box
  •  Conclusion: your hypothesis is falsified, there might be something in the box! (note: might, see below)
So you’ve falsified your hypothesis! What next? Overpriced plastic theme park? Not quite. There are a number of ideas in the realm of reason that you need consider:
  1. You don’t know for a fact that the empty box is exactly the same construction as the experimental box since you didn’t get a chance to investigate the experimental box in detail. If it’s made out of a lightweight but very tough material it might be able to repel shotgun shot better than your control (which is made out of reinforced cardboard).
  2. There might be different amounts of shotgun shot in each shot. You probably need to investigate, either by counting the number of shot in each shell or by shooting a bunch of boxes and doing statistics. Oh, you only have one mystery box? Well too bad, probably a non-destructive method might be better.
  3. Maybe shotgun shot passes as easily through whatever is in the box as it does through air (or at least they’re close enough that you can’t tell the difference with your, admittedly, blunt measurement method).
  4. Maybe, due to random variations you see a difference where there really isn’t one. See point 2 above.
  5. Maybe…. you get the idea
So what is the conclusion? The conclusion is that you’ve eliminated a number of reasonable alternative pathways and that your hypothesis has been falsified- though with only one example no reviewer is going to believe that. So do your experiments on three or more mystery boxes and resubmit your story. At that point you can safely say that the hypothesis is reasonably falsified. Of course, there are still things that could have happened to give the results that don’t involve anything being in the box.
Stay tuned! Next post: How does computational biology work as science?


Prometheus the creationist

This discussion probably contains spoilers, not really of the specific things that happen in the movie, but of the overall ideas. So if you haven’t seen it and are likely to watch it, keep that in mind.

Enough of the plot to get you going if you haven’t seen the movie. It’s the year 2093. Two scientists (maybe archaeologists? not really sure- but they sure is smart) discover an ancient cave drawing that (along with a bunch of other cheap knock-offs from around the globe spanning cultures and centuries) point them to a distant planet. A mysterious rich guy funds a spaceship (Prometheus!) to explore the planet and test the scientists’ hypothesis. Possibly the old guy has an ulterior motive or two? The hypothesis is that aliens created humans and they’re from this planet (see below for discussion). The spaceship reaches said planet, lands- the explorers start poking around and making the standard horror movie mistakes. Mayhem ensues.

So I loved it. Great action, great pacing, good characters, some suspense, a lot of grossness, a pretty self-consistent plot all-in-all (see my points below for how it doesn’t agree with science). However, there was one major problem for me that is hard to ignore. Why did the writers, as one character puts it in the start of the movie, “just throw away 300 years of [understanding of] Darwinian evolution?” The science part of the science fiction premise for the movie is on one hand a great one: humankind’s search for their own origins. As a scientific allegory this really works, and the addition of the extra layer, the human-created robot David, in the movie really works well. On the other hand, WTF!?! It makes really no sense in terms of anything scientific. Here are my main points:

  1. If the aliens planted us on Earth at sometime long enough ago to start the human race, how did they then communicate to different human cultures a pictogram map to the ‘home’ world? I guess you could argue that there were things that either weren’t known to science in the year 2093 or were known but not discussed in the context of the movie. Namely, that the aliens then returned and communicated with these various cultures- the logical resolution to this problem. We, as viewers, can fill this point in ourselves in a logical fashion.
  2. Here’s my big one: if aliens planted us on Earth at some point in the past, and our DNA is identical to their DNA, this raises a number of paradoxes that just don’t make even Sci-Fi sense:
    1. If we were planted along side a bunch of Earth-evolved animals how is it that we are so close in terms of DNA with other of these species. Mouse DNA and human DNA is something like 90% similar on the gene level. The aliens would have had to match that when they created us, but the Prometheus story is that they created us out of their DNA. Doesn’t make sense.
    2. If we were planted on Earth before the emergence of any animal life, then all other animal life (and plant too probably) would have to be derived from us so that everything is similar. Completely doesn’t fit with the fossil record, or any phylogeny of relatedness between species. We are not the progenitors of all other life on Earth- much closer to the reverse holds.
    3. So it seems that in the context of the movie either evolutionary science is all wrong, or that our understanding of the fossil record is all wrong or both. Or that there was something immensely complicated and unreasonable (in terms of amount of effort) that the aliens did to make things look like they do.
    4. The movie would make more sense and be more believable if there were some mention somewhere of these problems and some reasonable solution. For example, that scientists had made some big discovery sometime in the time between 2012 and 2093 that explained it all that is talked about in the movie but never explained, but the movie would have been better for it.
    5. Without such an explanation the “science” underlying it looks an awful lot like creationism with a big dose of intelligent design. The fossil record was manufactured (or we have consistently and massively misinterpreted it in many, many, many, independent ways), evolution is untrue or doesn’t work the way we think it does, and we have to look beyond evolution for the answers to our existence (which is, in my opinion, a total cop out). So that leaves me, on one level, fundamentally disturbed and unsatisfied by the movie.
    6. Nowhere in the movie is it ever explained why the scientists in the movie believe that this alien race “created” us. It seems more plausible to hypothesize that we descended from them, maybe from a marooned spaceship or something, given that their DNA is ‘identical’ to ours. I don’t really have a problem with this either way (except for the points I raise above), but it would have been nice for them to acknowledge this rather major plot assumption that doesn’t seem to have much basis. I’ll just chalk this one up to some scene that landed on the cutting room floor.
    7. Here’s a minor nitpicky science point: when they use carbon dating on the alien planet to determine the age of remains they discover. That’s really unlikely to work unless the particular distribution of carbon isotopes on Earth is found everywhere else in the universe. But that was pretty forgivable and more fun to find than annoying.

As I said at the start, I loved the movie. Thought it was great. It works really well as a science-based allegorical story about humans searching for meaning, and their own origins. No problems there. And I agree that the basic science plot is compelling: aliens created us and they planted us on Earth. It’s an idea that’s been explored before many times (though it would always suffer from the points I raise above). The big problem is that I just don’t understand why they didn’t address this, somewhere. Or maybe I’m asking too much of a sci-fi horror movie. Could be.


View the trailer here.