Decision Errors
Chapter 14
Math 215
Math 215
CPR Study
- Research question: Do blood thinners affect survival rate in heart attack patients that have received CPR?
cpr1 dataset- 2 variables
- group: treatment (received blood thinner) or control (did not)
- outcome: died or survived (for at least 24 hours)
- 90 patients (40 treatment, 50 control, randomly assigned)
Hypotheses
- Blood thinners can be administered to treat a clot that is causing a heart attack
- CPR can cause internal injuries
- Blood thinners can make it more difficult for these injuries to heal
- Do blood thinners affect survival in a positive or negative way?
- Alternative hypothesis reflects the fact that we don’t have expectations about the direction of the relationship
Two-sided hypothesis test
\(H_0\): Blood thinners do not affect survival rate. \(p_T-p_C = 0\)
\(H_A\): Blood thinners affect survival rate. \(p_T-p_C \neq 0\)
Example of one-sided hypothesis test
\(H_0\): Blood thinners do not affect survival rate. \(p_T-p_C = 0\)
\(H_A\): Blood thinners increase survival rate. \(p_T-p_C > 0\)
Results (EDA)
Difference in proportions
- Success: outcome = “survived”
- Statistic of interest: difference in proportions \[\hat{p}_T-\hat{p}_C\]
- Observed difference: \[\frac{14}{40}-\frac{11}{50}=0.13\]
Hypothesis test
- Technical conditions met for using a normal approximation
- SE = 0.0955 (more on estimating this in Ch. 17)
- Null distribution: \(N(0, 0.0955)\)
- We will use significance level \(\alpha=0.05\)
Density function for N(0, 0.0955) with tails shaded beyond 0.13.
- p-value is probability that statistic is at least as extreme as observed value (0.13)
- 2-sided test: at least as extreme means \(\geq 0.13\) or \(\leq -0.13\)
- Since the null distribution is symmetric, the p-value will be twice as large for a 2-sided test
- Since the p-value is greater than 0.05, we do not reject the null hypothesis
- It is plausible that there is no difference in survival rates between the two groups
Decision Errors
- There are two types of errors we can make in a hypothesis test
- Type 1 Error occurs if we conclude that the null hypothesis is false when it is not (false alarm)
- Type 2 Error occurs if we fail to reject the null hypothesis even though it is false (missed opportunity)
Example of Type 1 and Type 2 Errors
In the study on the blood thinners, we concluded that there is no significant difference in survival rates. If in reality the rate of survival is significantly different between two groups that would mean that we committed Type 2 Error
If the p-value of the test were lower than the significance level and we rejected the null hypothesis, but, in reality, the survival rates are the same - that would mean we committed a Type 1 Error
Note that once a conclusion is made (based on p-value or z-score) we can possibly commit only one type of error
Summary of Type 1 and Type 2 Errors
Controlling Type 1 Errors
- Type 1 Error is typically considered more severe
- The probability of making a type 1 error (assuming the null is true) is equal to the significance level
- Can reduce \(\alpha\) to make type 1 errors less likely
Controlling Type 2 Errors
- There is a trade-off between the two types of errors
- Decreasing probability of type 1, increases the probability of type 2
- Power is the probability of rejecting the null hypothesis if the alternative is true
- \(Pr[Type\:2\:Error]= 1- Power\)
- Higher power reduces the chance of making a type 2 error
- Power is related to effect size (easier to detect larger effects), sample size (larger sample results in more power), among other things