Debugging with the Scientific Method
One of the best techniques I’ve run across for debugging software is the Scientific Method.
The Scientific Method for Debugging
It’s actually pretty simple, but I’m always amazed at how quickly it can bring you to the root of a problem. Basically, it goes like this:
- Enumerate the evidence you’ve seen regarding the issue.
- Before changing any code, take a guess at what might be happening. It doesn’t have to be a good guess—it just has to be something you can test. This is called your hypothesis.
- Come up with a way to test your hypothesis. You want something that can give you a “yes” or “no”.
- If your hypothesis is correct, you’ve found the issue. Fix it and move on. If your hypothesis is incorrect, log this fact as the next piece of evidence and then come up with a new hypothesis based on all of the evidence, including this.
For a typical issue, it takes me 2 to 3 hypotheses to get to the root cause. For tougher issues (especially those that involve recursion or graphs), it takes me 5 to 7. In either case, it’s a great way to prune off areas to search. It is by far the most effective way of debugging that I’ve ever found. I’ve often noticed that issues typically requiring a day (or more) of debugging can be dispatched in an hour or so using this technique.
Modification using TDD
One powerful modification is using test-driven development with the Scientific Method. This is particularly useful when it takes many steps to duplicate the issue using the application. It’s even more useful when duplicating the issue requires you to delete and re-add data.
Here, the first step is to duplicate the issue in an automated test. This is the hard part, but the part that will save you the most time. Once you have this going, the tests of your hypotheses will be the same as your TDD tests. You add tests to check your hypotheses until you’ve proven your hypothesis correct. A bonus is that once you’re done, you’ll have some regression tests in place that actually exercise something that broke.
Believe it or not, it’s kind of fun to debug this way. You feel like a scientist wearing a white lab coat and jotting notes into a log book (I assume you think that’s fun 🙂 ). You set up experiments to see if you can trap bugs. They can be surprisingly evasive, but the scientific method always catches them in the end. Bwahahaha!