Testing of Hypothesis: Lecture note 2- Level of the test Level of significance Power of the test

From the previous lecture, it is clear that to make a certain decision about the null hypothesis on the basis of the data we need a specific decision rule. In this decision-making process, we may commit two kinds of errors, Type I error and Type II error.

To make a good decision we must restrict these two errors, but we can not minimize both errors simultaneously (why?? I will explain it in the next lecture note).

We need a well-defined decision rule or test rule so that P(Type I error) < 0.05 (or 0.01) and P(Type II error) is minimum.

We define P(Type I error) as the level of the test.

The upper bound of the P(Type I error ) is called the level of significance. The usual values of the level of significance are 0.05 and 0.01.

The power of the test is defined as 1- P(Type II error).

Thus we need such decision rules where the level of the test is under the level of significance and the power of the test is maximum.

Before getting into the calculation of the level of the test and the power of the test, we need to better understanding of decision rule.

Suppose we are about to judge the fairness of the dichotomous or binary event. We suspect that the success ratio is a little bit high. To make it easy we can think of this dichotomous or binary event as the fairness of a usual coin.

Our null hypothesis, Ho: the coin is fair i.e. p = 1/2, (here p = P(H) or P(success))

Our alternative hypothesis, H1: the coin is biased towards the head i.e. p > 1/2

For now, we will toss the coin 4 times (it means we repeat the experiment 10 times) to get the desired data.

Our decision rule (intuitive): If we get more than or equals to 3 heads then we will reject the null hypothesis.

With this decision rule, we are basically dividing the sample space into two parts, one is the rejection zone another one is the acceptance zone or not the rejection zone.

How?

If we toss the coin 4 times the sample space is {HHHH, HHHT, HHTH, HTHH, THHH, HHTT, HTTH,THTH, HTHT, THHT, TTHH, TTTH, TTHT, THTT, HTTT, TTTT }. Now if we get any one of the outcomes from the highlighted portion of the sample space, where the number of heads more or equals 3, we will reject the null hypothesis. This portion of the sample is the rejection region, the technical name of it is the Critical region. If we get any outcome outside of this highlighted portion or the critical region, we will not reject the null hypothesis.

Thus the moral of the story is, a decision rule creates a critical region. If we get the data or observation from the critical region, we reject the null hypothesis.