## Blogs

Sign-up to receive the latest articles related to the area of business excellence.

## Hypothesis Testing Overview

View All Blogs

But such things do happen if we are not careful in our analysis. For example, we may have heard about famous stock pickers who can consistently pick good quality stocks in the long-run. We may even consider investing our hard earned money with such agents. There may be great stock pickers out there (such as Warren Buffet) but let me show you have you could claim to be such a picker. Let’s say that there are 5 quality stocks A, B, C, D and E. You invest a modest amount of money in each of the 5 stocks. Let’s say in one year stock A does exceptionally well. You now discard the other stocks and invest again next year in F, G, H, I, and J. Let’s say that this year stock G does exceptionally well. You again discard the rest and continue to do so every year. In about 10 years, you can claim that I was so good at investing that first year I picked stock A, second year I picked stock G, third year I picked stock P and so on and if you had invested with me 10 years ago, your cumulative return would have been 5000%. Here, we are only looking at the successes and not at the failures – so we encounter what is called a survivor bias. We cannot draw good conclusions if we only look at successes without also looking at the failures.

Similarly, when we are trying to solve a business problem, say P, and want to find out if R is a root cause. We need to collect some data to answer this question. When usually work with sample data and there are two types of errors we can make. Based on our data analysis, we may incorrectly conclude that R is not a root cause when in fact it really is a root cause and secondly, we may conclude that R is a root cause when in fact it is not the root cause of our problem. Ideally, we don’t want to make these types of mistakes in our analysis because these mistakes can be expensive. If R was a root cause and we miss it, we cannot put solutions in place to solve the real problem. If R was not a root cause and we picked it, we may be wasting valuable company resources addressing issues of minor relevance.

That is where hypothesis testing comes to our rescue – done properly, we can avoid the types of mistakes we discussed above. The power of statistics comes into play when we can make good conclusions with limited amount of data! There are a lot of different hypothesis tests out there and each test is applicable to a different situation, each have their own assumptions and requirements and if we want to do proper analysis, we need to pick the right hypothesis test. The hypothesis test we pick depends on the type of data we have for our primary metric or the main question we are trying to answer (Y) and the type of causes (X). For example, if we want to determine whether using software A or software B has an impact on our time to complete a task. The primary metric (Y) would be the time to complete the task (which would be continuous in this case) and the cause would be the software (discrete in this case with 2 possible values). The following chart will help you pick the right hypothesis test for your problem. Selecting the right test is just one step to solving the problem. Other steps include formulating the right hypothesis, selecting the right data for analysis, drawing the right conclusions etc. These topics will be covered in a separate blog.

The objective of this blog is to highlight that in the real-world we deal with limited set of sample data and in order to make good business decisions, we need to use the right statistical tools (hypothesis testing in this case). Of course, you may be wondering why go through all this complex stuff when simple tools would be enough to solve the problems. You would be right depending on the problem we are attacking. However, there are some classes of problems where you must perform proper data analysis and draw the right conclusions. Problems where too much money is involved, problems that require significant amount of time and resources to solve, or that entail significant risk. In these cases, we cannot just "hope" or "guess" through intuition that we have the right answer. If we do that, we might just as well be the mystic who can predict the future.