Decision Errors

IMS2 Ch. 14
Math 115

Yurk

CPR Study

Research question: Do blood thinners affect survival rate in heart attack patients that have received CPR?
cpr data set is available here
2 variables
- group: treatment (received blood thinner) or control (did not)
- outcome: died or survived (for at least 24 hours)
90 patients (40 treatment, 50 control, randomly assigned)

Blood thinners can be administered to treat a clot that is causing a heart attack
CPR can cause internal injuries
Blood thinners can make it more difficult for these injuries to heal
Do blood thinners affect survival in a positive or negative way?
Alternative hypothesis reflects the fact that we don’t have expectations about the direction of the relationship

Two-sided hypothesis test

Example of one-sided hypothesis test

group	died	survived	total
control	39	11	50
treatment	26	14	40
total	65	25	90

Technical conditions met for using a normal approximation for the null distribution
Since we calculated \(Z\), we can use the standard normal distribution \(N(0,1)\)
We will use significance level \(\alpha=0.05\)

As with a 1-sided test, the p-value is the probability of observing a value of the statistic that is at least as extreme as the observed value, assuming the null hypothesis is true
With a 2-sided test, extreme values are in both tails of the null distribution

To calculate the p-value for a 2-sided hypothesis test we start by calculating two areas
- The area to the left of the observed statistic
- The area to the right of the observed statistic
The p-value is twice the smaller area

Density function standard normal distribution with left tail shaded below Z=1.36.

Density function standard normal distribution with right tail shaded above Z=1.36.

p-value is \(2\times0.0869 = 0.1738\)
Since the p-value is greater than 0.05, we do not reject the null hypothesis
It is plausible that there is no difference in survival rates between the two groups
Note: The p-value is twice as large for a two-sided test than for a one-sided test

With a symmetric null distribution (like the normal distribution) that is centered at 0, the two tails are symmetric
Thus, for example, the p-value is also equal to the sum of the areas in the two tails that are to the left of -1.36 to the right of 1.36

Density function standard normal distribution tails shaded below Z=-1.36 and above Z=1.36.

There are two types of errors we can make in a hypothesis test
Type 1 Error occurs if we conclude that the null hypothesis is false when it is not (false alarm)
Type 2 Error occurs if we fail to reject the null hypothesis even though it is false (missed opportunity)

Type 1 Error is typically considered more severe
The probability of making a type 1 error (assuming the null is true) is equal to the significance level
Can reduce \(\alpha\) to make type 1 errors less likely

There is a trade-off between the two types of errors
Decreasing probability of type 1, increases the probability of type 2
Power is the probability of rejecting the null hypothesis if the alternative is true
Higher power reduces the chance of making a type 2 error
Power is related to effect size (easier to detect larger effects), sample size (larger sample results in more power), among other things

A significance level of \(\alpha\) for a 2-sided hypothesis test corresponds to a confidence level of \(1-\alpha\)
E.g., \(\alpha = 0.05\) corresponds to a 95% confidence interval
If we would reject the null hypothesis at \(\alpha=0.05\), then the 95% confidence interval would not contain the null value
Conversely, if we calculate the 95% confidence interval first and find that it does not contain the null value, then we would reject the null hypothesis at \(\alpha=0.05\) with a two-sided test

In the CPR study, we did not reject the null hypothesis at \(\alpha=0.05\) (p-value = 0.1738)
Even though we did not calculate a confidence interval for the difference in survival proportions, we know that the 95% confidence interval would contain 0 (the value under the null hypothesis)