logistic_spec <- logistic_reg() %>%
set_mode("classification") %>%
set_engine("glm")
full_fit <- logistic_spec %>%
fit(Direction ~ Lag1 + Lag2 + ... + Volume, data = Weekly)Chapter 4 Applied Problems: Guided Instructions
Introduction
Use the classification approaches introduced in the Chapter 4 lab to complete these problems. Use parsnip, recipes, and workflows when appropriate, and produce all graphs with ggplot2.
The guidance identifies useful functions, variables, and modeling choices, but it does not provide complete solutions. Some short code patterns are included for unfamiliar constructions; you must complete the missing objects and arguments. The displayed R fragments are not evaluated when this instruction file is rendered.
Useful packages include tidyverse, tidymodels, ISLR2, corrr, and discrim. The required engines may also need the MASS, klaR, and kknn packages installed.
Problem 1: Weekly market direction
Complete Problem 13 from Chapter 4 of ISLR (p. 193). The response Direction records whether the market increased or decreased during a week.
Part 1(a): Explore the Weekly data
Why this matters. Exploratory analysis identifies class imbalance, time trends, and promising predictors before a classifier is fitted.
Begin with glimpse(Weekly), summary(Weekly), and count(Direction). Then investigate the quantitative variables. One useful correlation pipeline is Weekly %>% select(-Direction) %>% correlate(). Store the result and consider displaying it with rplot().
Produce focused graphs of Volume versus Year and the proportions of Up and Down weeks across years. A volume graph can begin with ggplot(aes(x = Year, y = Volume)).
You can optionally add scale_y_log10() to a second version of the volume plot to make proportional changes clearer. Keep the ordinary-scale graph too, so the two views can be compared. The logarithmic scale is a visualization choice here, not a required transformation of the data.
For the direction graph, you may convert Year to a factor and use geom_bar(position = "fill"). Describe the clear patterns, distinguishing trends over time from variables that might predict Direction.
Part 1(b): Logistic regression with all lag predictors
Why this matters. This model tests whether recent returns or trading volume contain information about the current week’s direction after the other predictors are accounted for.
Fit logistic regression with Direction as the response and Lag1 through Lag5 plus Volume as predictors. Do not include Year or Today.
Create the specification with logistic_reg(), set_mode("classification"), and set_engine("glm"). A direct parsnip fit may be used:
Replace ... with the missing lag variables. Use tidy() or extract the underlying engine and apply summary(). Identify statistically significant predictors at a stated significance level. Interpret signs as changes in log odds, not direct changes in probability.
Part 1(c): Confusion matrix on the fitting data
Why this matters. Coefficients describe estimated relationships, while a confusion matrix shows how often the classifier is correct and which type of error it makes.
Attach predictions with fitted_model %>% augment(new_data = Weekly) and save the result. Apply conf_mat(truth = Direction, estimate = .pred_class) and accuracy() to that prediction tibble.
Explain both off-diagonal cells. Compare the model’s accuracy with the accuracy of always predicting the more common class, which can be found from count(Direction). Remember that this is training accuracy and may be optimistic.
Part 1(d): Logistic regression with held-out years
Why this matters. Testing on later years gives a more honest indication of performance on future market data.
Create a training set for 1990-2008 and a test set for 2009-2010. The filters should use conditions like Year <= 2008 and Year >= 2009. Verify the year ranges and row counts.
Fit Direction ~ Lag2 using only the training data. Pass the test tibble to augment(new_data = ...), then calculate its confusion matrix and accuracy. Do not refit the model on the test data.
Part 1(e): Linear discriminant analysis
Why this matters. LDA supplies a distribution-based classifier with a linear boundary, providing a direct comparison with logistic regression.
Repeat Part 1(d) with Lag2 as the only predictor and the identical split. Use discrim_linear(), classification mode, and the "MASS" engine. Only the specification should change; reuse the same fitting, augmentation, and evaluation pattern.
Part 1(f): Quadratic discriminant analysis
Why this matters. QDA allows the classes to have different variances and a more flexible boundary, but that flexibility can increase test error.
Repeat the analysis using discrim_quad(), classification mode, and the "MASS" engine. Keep Lag2, the training rows, and the test rows unchanged so that the comparison remains fair.
Part 1(g): K-nearest neighbors
Why this matters. KNN classifies a week from nearby training observations instead of assuming a probability model.
Fit KNN with \(K=1\) using Lag2. Build the specification with nearest_neighbor(neighbors = 1), classification mode, and the "kknn" engine. Evaluate it on the same test set.
With one predictor, scaling is not important. It becomes essential when several predictors are combined in Part 1(j). Explain why \(K=1\) creates a flexible, high-variance classifier.
Part 1(h): Naive Bayes
Why this matters. Naive Bayes supplies another distribution-based classifier. With one predictor, its conditional-independence assumption is not restrictive.
Use naive_Bayes(), classification mode, and the "klaR" engine. Fit Direction ~ Lag2 on the training rows. Make the usekernel choice explicit; using usekernel = FALSE gives a normal-density version.
Evaluate the same held-out observations and report the confusion matrix and accuracy.
Part 1(i): Compare the five classifiers
Why this matters. Accuracy alone can conceal very different error patterns. A common prediction table ensures that all methods are evaluated on the same observations and with the same event-class definition.
Compare logistic regression, LDA, QDA, KNN, and naive Bayes using accuracy, sensitivity, specificity, and ROC curves. Before calculating sensitivity and specificity, identify the event class. If Down is the first factor level, use event_level = "first" consistently.
Place the fitted models in a named list and collect predictions with an anonymous function:
model_list <- list(
"Logistic regression" = ..., "LDA" = ..., "QDA" = ...,
"KNN" = ..., "Naive Bayes" = ...
)
all_predictions <- model_list %>%
imap_dfr(function(fitted_model, model_name) {
augment(fitted_model, new_data = Weekly_test) %>%
mutate(model = model_name)
})Use metric_set(accuracy, sens, spec), group the prediction tibble by model, and supply Direction, .pred_class, and the event-level argument. For ROC curves, use the probability for the event class–for example, .pred_Down–rather than .pred_class.
Explain why a curve nearer the upper-left corner is better, and use class-specific metrics as well as accuracy when selecting a model.
Part 1(j): Explore alternative models
Why this matters. The first comparison changes the method while holding the predictor fixed. This part asks whether additional information or a better flexibility choice improves held-out predictions.
Begin with a manageable candidate set such as Lag1 + Lag2 + Volume. Consider the following options:
- optionally apply
step_log(Volume)because volume is strongly right-skewed; - try a scientifically reasonable interaction if a plot suggests one;
- apply
step_normalize(all_numeric_predictors())before KNN; and - compare \(K=1,3,5,\) and \(10\).
Use a recipe when preprocessing is needed. Give each workflow a distinct name so models are not overwritten. Report the final method, predictors, transformations, value of \(K\) when relevant, test confusion matrix, and metrics. Compare the result with the simpler Lag2 models.
Problem 2: High versus low automobile fuel economy
Complete Problem 14 from Chapter 4 of ISLR (p. 194) using the modifications below.
Part 2(a): Create mpg01
Why this matters. Dividing fuel economy at its median creates the binary response needed for classification and produces groups of roughly comparable size.
Compute the median with a sequence such as Auto %>% summarize(median_mpg = median(mpg)) %>% pull(median_mpg). Store the result before using mutate().
Create mpg01 with if_else(mpg > mpg_median, ..., ...). Assign class 1 above the median and class 0 otherwise. Convert it to a factor with levels ordered 0, then 1, and verify it with count(mpg01).
Part 2(b): Graphical exploration
Why this matters. Graphs identify predictors that separate the two classes and prevent an unnecessarily large model.
Concentrate first on cylinders, displacement, horsepower, weight, and acceleration. Compare the four quantitative measurements with side-by-side boxplots by mpg01. For cylinders, convert the values to a factor and consider a proportional bar chart using position = "fill".
A compact option for the quantitative graphs is to select mpg01 and the desired predictors, apply pivot_longer(), and facet the resulting boxplots with free y-scales. Identify a reasonable common predictor set for the classifiers. Do not include the original mpg, since it directly determines mpg01.
Part 2(c): Training and test sets
Why this matters. A time-based split tests whether patterns learned from earlier automobile designs generalize to later model years.
Use approximately 75% of the data for training. A reasonable cutoff is 1979: use year < 79 for training and year >= 79 for testing. Verify the row counts, year ranges, and mpg01 class counts instead of assuming the split is appropriate.
Common recipe for Parts 2(d)-2(h)
Use cylinders, displacement, horsepower, and weight as a reasonable common predictor set. The following recipe may be completed and reused:
auto_rec <- recipe(
mpg01 ~ cylinders + displacement + horsepower + weight,
data = Auto_train
) %>%
step_normalize(all_numeric_predictors())Normalization is essential for KNN and does not prevent fitting the other requested classifiers. The formula deliberately excludes mpg and name.
Part 2(d): Linear discriminant analysis
Why this matters. LDA provides an interpretable baseline with a linear decision boundary and a pooled within-class covariance structure.
Use discrim_linear(), classification mode, and the "MASS" engine. Add the common recipe and model specification to a workflow, fit only on Auto_train, and predict Auto_test.
Report the confusion matrix, accuracy, sensitivity, specificity, and test error, where test error is \(1-\text{accuracy}\). State that factor level 0 is treated as the event if event_level = "first" is used.
Part 2(e): Quadratic discriminant analysis
Why this matters. Comparing QDA with LDA shows whether separate covariance structures improve classification enough to justify a more flexible boundary.
Repeat Part 2(d), changing the model specification to discrim_quad() with the "MASS" engine. Keep the recipe, training set, and test set fixed.
Part 2(f): Logistic regression
Why this matters. Logistic regression models class probability directly and provides coefficients that can be interpreted through log odds or odds ratios.
Use logistic_reg(), classification mode, and the "glm" engine with the same recipe and split. Use .pred_class for the confusion matrix and the appropriate .pred_0 or .pred_1 column for ROC analysis.
Part 2(g): Naive Bayes
Why this matters. Naive Bayes demonstrates that a simple probability model can classify well even when the conditional-independence assumption is only approximate.
Use naive_Bayes(), classification mode, and the "klaR" engine. State whether usekernel is TRUE or FALSE; using FALSE provides a normal-density comparison. Fit and evaluate it with the common recipe and split.
Part 2(h): K-nearest neighbors
Why this matters. KNN can create nonlinear boundaries, but it is sensitive to predictor scales and to the number of neighbors.
Fit \(K=1,3,5,\) and \(10\). Each specification should use nearest_neighbor(neighbors = ...), classification mode, and the "kknn" engine. Reuse the normalized recipe and compare test accuracy, sensitivity, and specificity. Explain how increasing \(K\) produces a smoother, less flexible classifier.
Part 2(i): Compare all models
Why this matters. A single prediction table prevents accidental differences in test cases, event class, or metric definitions from affecting the comparison.
Place the fitted LDA, QDA, logistic-regression, naive-Bayes, and KNN workflows in a named list. Use the anonymous imap_dfr() pattern from Problem 1 to collect predictions on Auto_test.
Group by model before applying metric_set(accuracy, sens, spec). If level 0 is the event, use event_level = "first" and .pred_0 for roc_curve() and roc_auc().
Create a compact comparison table containing accuracy, sensitivity, specificity, and test error. Identify the preferred method using the metric balance, ROC performance, complexity, and assumptions–not accuracy alone. When two methods perform similarly, prefer the simpler or more interpretable method unless there is a clear reason not to.
Submission reminder
Clearly distinguish training data from test data. Include the requested confusion matrices, metrics, and ROC curves with concise interpretations. State the event class used for sensitivity, specificity, and ROC calculations.