The scientific method (as imagined by a biologist-turned-computational-biologist)

“I have approximate answers, and possible beliefs, and different degrees of certainty about different things, but I’m not absolutely sure of anything”- Richard Feynman

“In science, ‘fact’ can only mean ‘confirmed to such a degree that it would be perverse to withhold provisional assent.'” – Stephen Jay Gould

“Computational biology is just biology”- me.

So the genesis of this post is that there are some ‘real biologists’ or other ‘real scientists’ who believe that computational biology is sitting around making things up on computers willy-nilly, or randomly wandering through data to come to some conclusion, or writing code to solve problems that no one (no biologist) actually cares about, or some other such endeavor that doesn’t count as science. In many cases this is simply a field-based cultural perception that they don’t really give a thought to (i.e. they aren’t trying to dis us actively) but others that think a lot about these kinds of issues seem to campaign against us. Of course, there are many examples of computational analysis of biological data that do fall into the categories of non-science, just as there are many scientists who do experiments without appropriate attention to the scientific method and thus fall into the same boat.

Generally, what is the scientific method?

First, here are the steps I think of in the scientific method:

1. Hypothesis formation: in which you determine what the question is that you want to ask. This is generally developed from previous experiments (see step 5 below), but can also be from ‘exploratory’ experiments and/or analyses. Generally, ‘exploratory’ simply means that the hypothesis being tested in the experiment is very broad.

2. Formulation of experiment: in which you figure out how you will ask the question. In science this is to falsify your hypothesis- that is, given this hypothesis how would I show that it ISN’T true. This ALWAYS includes controls- that is, portions of the experiment that should turn out negative and/or positive by your evaluation method.

3. Execution of experiment: do the experiment.

4. Evaluation of initial hypothesis: examine the results carefully. Compare the experimental results against the results from your control experiment. Did the control experiments work? If not, something is wrong with your experiment. If they did, do the results of your experiment differ significantly from the controls? Are the results clear enough to either falsify your hypothesis (generally considered a negative result) or not falsify your hypothesis? No hypothesis is ever ‘proven’, only proven false or not disproven.

5. Formulation of further experiments. If your hypothesis still survives then how else might you falsify the hypothesis. Generally there should be multiple ways that you can think of to falsify your hypothesis- that is, multiple reasons for the results of your experiment that are not the idea put forth by your hypothesis. You have to test these too to try to falsify your hypothesis. If you don’t, reviewers will ask for you to do it- or at least they should. You should not rule a hypothesis ‘supported’ until you have investigated, and discarded several alternative pathways that might have generated the results you’ve seen. So these further experiments may be inverted from your original hypothesis in that you are proposing a new hypothesis that your result is due to something else, then trying to falsify this alternative. Go back to step 2.

6. Reformulation/modification of hypothesis anew. Alternative 1: Your hypothesis has survived! Congratulations, you move to the next step- given that your hypothesis is true (note that you actually have not SHOWN that your hypothesis is true, only that it is not false by a limited number of alternative experiments [see step 5]- which is why I say “given that your hypothesis is true”, not “since your hypothesis is true”), what is the next question that is suggested? Are there implications? Have you seen anything else in the results that seems funny or interesting or hard to explain? Maybe that can be interpreted in the context of a new hypothesis and you can discover new things! Go to step 1. Alternative 2: Your hypothesis was falsified. Too bad, that’s science. Is the original phenomenon or observation you were trying to investigate still interesting? Can you think of an alternative hypothesis that is testable given your experimental and/or data limitations? The phenomenon has to be happening for some reason and you can determine it with enough hard work so get back at it. Go to step 1.

A toy example of how the scientific method works

You’ve found a box. It’s small enough to pick up and sealed closed. Your hypothesis is that the box is completely empty.

Round 1:

  • Hypothesis: there’s nothing in the box
  • Experiment: shake the box
  • Control: shake a box you know is empty
  • Results: there is no sound and you don’t hear anything rattling around just like the positive control box
  • Conclusion: the hypothesis is supported, there is nothing in the …. wait, wait, wait, wait! What if there’s something in the box that doesn’t rattle?
Round 2:
  • Hypothesis: there’s nothing in the box (but now you’ve eliminated the possibility that it rattles)
  • Experiment: weigh the box
  • Control: weigh the empty box
  • Results: you can detect no difference in weight between your ‘experimental’ box and your control box
  • Conclusion: the hypothesis is supported, there is nothing in the….. hmmmmmm….. OK. What if what’s in the box doesn’t weigh much at all. Actually what if it weighs little enough to fall in the margin of error for the scale (or other method) you’re using to compare the two?
Round 3:
  •  Hypothesis: there’s nothing in the box (but now if there is something there it doesn’t rattle and doesn’t weigh much)
  • Experiment: take a shotgun and shoot the box
  • Control: shoot an empty box (OK- these are simple controls, but you get the idea)
  • Results: you see that significantly fewer holes in the exit side of the ‘experimental’ box relative to the control box
  •  Conclusion: your hypothesis is falsified, there might be something in the box! (note: might, see below)
So you’ve falsified your hypothesis! What next? Overpriced plastic theme park? Not quite. There are a number of ideas in the realm of reason that you need consider:
  1. You don’t know for a fact that the empty box is exactly the same construction as the experimental box since you didn’t get a chance to investigate the experimental box in detail. If it’s made out of a lightweight but very tough material it might be able to repel shotgun shot better than your control (which is made out of reinforced cardboard).
  2. There might be different amounts of shotgun shot in each shot. You probably need to investigate, either by counting the number of shot in each shell or by shooting a bunch of boxes and doing statistics. Oh, you only have one mystery box? Well too bad, probably a non-destructive method might be better.
  3. Maybe shotgun shot passes as easily through whatever is in the box as it does through air (or at least they’re close enough that you can’t tell the difference with your, admittedly, blunt measurement method).
  4. Maybe, due to random variations you see a difference where there really isn’t one. See point 2 above.
  5. Maybe…. you get the idea
So what is the conclusion? The conclusion is that you’ve eliminated a number of reasonable alternative pathways and that your hypothesis has been falsified- though with only one example no reviewer is going to believe that. So do your experiments on three or more mystery boxes and resubmit your story. At that point you can safely say that the hypothesis is reasonably falsified. Of course, there are still things that could have happened to give the results that don’t involve anything being in the box.
Stay tuned! Next post: How does computational biology work as science?


8 thoughts on “The scientific method (as imagined by a biologist-turned-computational-biologist)

  1. Very good visualization and explanation of how scientific theory works, Jason! I liked the “shooting the box” section, and I understand how much fun shooting at boxes is (I love target practice), but the practical side of me just says “open the box and look to see what is in it”. In that case, then you find another box with no obvious opening, so you have to repeat the hypothesis rounds again. Am I making any sense or just being silly?

  2. Nope- good point Heather. I intentionally avoided this possibility, but didn’t mention it in the explanation. For some reason you can’t or don’t want to open it up and look inside. Otherwise it makes not a great toy example. However, there are times in science when I’ve definitely had the feeling that someone has suggested, “Um, all that complicated stuff you’re doing is great, but why don’t you just open up the box and look inside?”. Generally a head-slapping moment.

  3. Pingback: The Scientific Method as it applies to Coding and Computational Biology | The Mad Scientist Confectioner's Club

  4. Pingback: How to write a scientific paper (part 1) | The Mad Scientist Confectioner's Club

  5. Pingback: Invisible science and how it’s too easy to fool yourself | The Mad Scientist Confectioner's Club

  6. Pingback: Fact from fiction: The scientific method is alive and well | The Mad Scientist Confectioner's Club

  7. Pingback: Job opening: worst critic. Better fill it for yourself, otherwise someone else will. | The Mad Scientist Confectioner's Club

  8. Pingback: Non-traditional scientists | The Mad Scientist Confectioner's Club

Leave a Reply

Your email address will not be published. Required fields are marked *