When using data to answer a question, your results might not be perfect. Lets think about two scenarios:
- Lets say you are trying to figure out if heads and tails are equally likely when you flip a coin. You flip a coin 6 times, and get THTTTH. That's odd... they should be the same. But numbers don't lie right? So you decide that your original hypothesis that they were equally likely is wrong.
- Having lost all confidence in your understanding of probability, you set out to test something you know is different, your expected winnings vs. expected losses when gambling. So you buy $100 worth of lottery tickets, and surprisingly receive $100 in winnings. You conclude that you are expected to break even in the long run.
In statistics, the first case is called "Type 1 error," deciding that the hypothesis that there is no difference (called the null hypothesis) is false when it is true. The second case is called "Type 2 error," deciding that there is no difference, when there is one.
Understanding that statistics can be misleading is important when making choices. But even more important is asking the right questions. The focus of this blog is avoiding this "Type 3 error," where you might get the right solution to a problem, but it is not really the problem you wanted to solve. The really unique thing about Type 3 error is that it can be eliminated completely by making sure you know which question you need to answer.
Could you give some examples of type three errors?
ReplyDeleteAlso, is that a name you came up with?
There are two major sources of type three errors. One is failing to include all the options, and the other is having the wrong criteria (In optimization terms these would basically be having the wrong constraints / variables and having the wrong objective).
DeleteIf you didn't include all the options, you might decide to buy a new car and not consider used cars. Another example would be deciding not to exercise since you hate running, instead of trying biking.
If you had the wrong criteria, you might buy the cheapest shampoo per ounce, instead of per use. Or you might decide to drive somewhere instead of flying since the odds of dying in a plane crash are higher than a car accident.
I did come up with it, and then a google search showed that I wasn't the first person to come up with it. But the other people I saw defined it roughly the same way :)