Inference: Logistic Regression

IMS2 Ch. 26
Math 219

Yurk and Martin

Diagnosing spread of cancer to lymph nodes

Can spread to lymph nodes from prostate cancer be diagnosed without surgery?
Data “lymph” from package HH
Deciding whether cancer has spread to lymph nodes is important for treating prostate cancer

The question is so critical in prognosis and treatment that it was customary to
- Operate on the patient for the sole purpose of examining the nodes
However, certain variables can be measured without surgery predicting spread to these nodes
Data using these variables was evaluated to see how well it predicted this spread

lymph ¹ data from 53 patients
7 variables, including

Variable	Description
`nodes`	Whether cancer spread to lymph nodes (0=“no”, 1=“yes”)
`X.ray`	Indicated by x-ray, 1 is serious, 0 is less serious
`stage`	Size and location of tumor, 1 serious, 0 less serious
`age`	Patient age at diagnosis
`acid.ph`	level of serum acid phosphatase

EDA

Sample sizes

X.ray	Stage 0	Stage 1
Serious	5	10
Less	21	17

Proportions of patients with cancer in lymph nodes

X.ray	Stage 0	Stage 1
Serious	0.400	0.900
Less	0.143	0.353

Logistic Model

Fit logistic model to predict whether patient has cancer in lymph nodes based on all predictors

lymph_mod <- glm(nodes ~ X.ray + stage + age + acid.ph,
           family = binomial, data = lymph)

tidy(lymph_mod)

# A tibble: 5 × 5
  term        estimate std.error statistic p.value
  <chr>          <dbl>     <dbl>     <dbl>   <dbl>
1 (Intercept)   0.446     3.41       0.131 0.896  
2 X.ray1        2.10      0.795      2.64  0.00833
3 stage1        1.76      0.747      2.36  0.0182 
4 age          -0.0703    0.0574    -1.22  0.221  
5 acid.ph       0.0222    0.0128     1.73  0.0828

The resulting model is

\[\begin{array}{rcl}\log\left(\frac{\hat{p}}{1-\hat{p}}\right) &=& 0.0446 \\ & + & 2.10\times X.ray1 \\ & + & 1.76 \times stage1 \\ & - & 0.0703 \times age \\ & + & 0.0222 \times acid.ph \end{array} \]

# A tibble: 5 × 5
  term        estimate std.error statistic p.value
  <chr>          <dbl>     <dbl>     <dbl>   <dbl>
1 (Intercept)   0.446     3.41       0.131 0.896  
2 X.ray1        2.10      0.795      2.64  0.00833
3 stage1        1.76      0.747      2.36  0.0182 
4 age          -0.0703    0.0574    -1.22  0.221  
5 acid.ph       0.0222    0.0128     1.73  0.0828

Two of the predictors (X.ray, stage) are statistically significant
Focusing on X.ray, it is unlikely (p-value < 0.05) to obtain a value of \(b_{1}\) as as far from 0 as 2.10 if X.ray is unrelated to nodes (\(\beta_{1}=0\)) and the model contains the other predictors

Manual Variable Selection

Multicollinearity is not a problem here, because the data are from a well-designed experiment
VIF are all near 1 (smallest possible value)

library(car)
vif(lymph_mod)

   X.ray    stage      age  acid.ph 
1.020128 1.056722 1.028123 1.062850

The coefficient for age is not significantly different from 0, so we drop it from the model

# A tibble: 4 × 5
  term        estimate std.error statistic p.value
  <chr>          <dbl>     <dbl>     <dbl>   <dbl>
1 (Intercept)  -3.58      1.18       -3.03 0.00247
2 X.ray1        2.06      0.778       2.65 0.00802
3 stage1        1.76      0.739       2.38 0.0175 
4 acid.ph       0.0206    0.0126      1.63 0.103

The coefficient for acid.ph is not significantly different from 0, so we drop it from the model
The remaining two coefficients are significant
Ironically, since the response is categorical and the remaining predictors (explanatory variables) are categorical
- We could analyze this data using a chi-squared test

# A tibble: 3 × 5
  term        estimate std.error statistic  p.value
  <chr>          <dbl>     <dbl>     <dbl>    <dbl>
1 (Intercept)    -2.04     0.610     -3.35 0.000802
2 X.ray1          2.12     0.747      2.84 0.00454 
3 stage1          1.59     0.700      2.27 0.0233

Spam

We would like to create a model that predicts whether an email is spam or not
email ¹ data from 3,921 emails

Variables

Variable	Description
`spam`	Whether email was spam (0 = “no”, 1 = “yes”)
`to_multiple`	Indicator: whether email was addressed to more than one recipient
`attach`	Number of files attached
`winner`	Indicator: whether the word “winner” appeared in the email
`format`	Indicator: whether email was written using HTML
`re_subj`	Indicator: whether subject started with “Re:”, “RE:”, etc.
`exclaim_mess`	Number of exclamation points in the message
`number`	Factor: whether there was no number, a small number (< 1 million), or a big number

About 9.4% of the emails were spam.

Logistic Model

# A tibble: 9 × 5
  term          estimate std.error statistic  p.value
  <chr>            <dbl>     <dbl>     <dbl>    <dbl>
1 (Intercept)  -0.335     0.111       -3.02  2.55e- 3
2 to_multiple1 -2.56      0.309       -8.28  1.21e-16
3 attach        0.197     0.0597       3.29  9.90e- 4
4 winneryes     1.73      0.325        5.33  9.79e- 8
5 format1      -1.28      0.130       -9.80  1.09e-22
6 re_subj1     -2.86      0.365       -7.83  5.06e-15
7 exclaim_mess  0.000229  0.000872     0.263 7.92e- 1
8 numbersmall  -1.07      0.141       -7.54  4.83e-14
9 numberbig    -0.424     0.202       -2.10  3.57e- 2

All but one of the predictors (exclaim_mess) is statistically significant

None of these variables is colinear because VIF of each is close to 1.

Email Classification

We would like to use the model to predict whether an email is spam or not
The model predicts the log-odds that an email is spam
One way to classify emails is to label an email as spam if the predicted probability exceeds 0.5
Most emails, including spam, have a low predicted probability of being spam
If we used a threshold of 0.5, only 1% of emails in the data would be classified as spam

Instead, we will use a lower threshold, 0.1
This allows us to classify more emails as spam
We expect to correctly classify a larger number of emails as spam
However, we also expect to incorrectly classify emails as spam that are not actually spam

Single-Predictor Model

We will compare the full model to a model that uses a single predictor (to_multiple)
We will use 4-fold cross-validation to evaluate the model performance

# A tibble: 2 × 5
  term         estimate std.error statistic   p.value
  <chr>           <dbl>     <dbl>     <dbl>     <dbl>
1 (Intercept)     -2.12    0.0562    -37.7  1.84e-310
2 to_multiple1    -1.81    0.297      -6.09 1.10e-  9

Cross-Validation Results

Confusion Matrix for Single-Predictor Model (holdouts)

	Predicted
Truth	Spam	Not Spam
Spam	355	12
Not Spam	2946	608

Overall accuracy: 0.246
Not spam, predicted correctly: 0.171
Spam, predicted correctly: 0.967

Confusion Matrix for Full Model (holdouts)

	Predicted
Truth	Spam	Not Spam
Spam	260	107
Not Spam	778	2776

Overall accuracy: 0.774
Not spam, predicted correctly: 0.781
Spam, predicted correctly: 0.708