Import Watershed Bacterial Community Data and Create Sequencing Depth Histogram
Introduction
In this assignment you will get you first opportunity to work with the Macatawa watershed bacterial community data! You will do the same data exploration, cleaning, and analysis that you did in Tutorial 4, but you will do it with the watershed data instead of the Erie data. Thus, you will produce a histogram of sequencing depths for the watershed sample, and you will produce filtered (clean) versions of the three data sets (shared, taxonomy, and meta) for use in future analyses.
Most of the code that we used in Tutorial 4 can be used with the watershed data (with appropriate name changes). There are a few things that will be different here, though. Some minor changes to the code are decribed below.
The biggest change in our analysis results from the fact that the watershed data set is MUCH larger than the Erie data set. Although you may be able to complete the analysis on your local computer (depending on your system resources), we will use the high performance computing cluster (HPC) to do the computations. Tutorial 5.5 introduced you to running R computations on the HPC. In that tutorial, you were provided with a Quarto document with all of the code you need to do the analysis for this assignment. We will assume that you have completed the tutorial, so that you have already created an RStudio project called watershedMacatawa to hold your code, data, and outputs on HPC, and that you have created links to the required raw data files.
You have two options for completing this assignment (see details below):
- Option 1: Modify the Quarto document from Tutorial 5.5 and run it on the cluster
- Option 2: Write your own Quarto document following Tutorial 4 and run it on the cluster
Differences between working with the Macatawa watershed and the Erie data sets
There are a few things we will need to do differently now that you are working with the watershed data instead of the Erie data. These are already reflected in the code for Tutorial 5.5. If you are working with your own Quarto file instead of the one provided in Tutorial 5.5, you will need to make sure that your code reflects these changes.
- Of course, the names of the objects you create should be different. Instead of using erie in the names use mac or macatawa
- There are a lot more columns in the Macatawa watershed metadata. We will use the following col_types when we import the data:
ccccDccccddccdcddddddddddddddddddddddddlddldcdciddciddddddddddd - There is some additional filtering that we need to do with the watershed metadata to remove some samples with problems and others that are used for quality control (see the code from tutorial 5.5)
- We do not need to convert the ‘CollectionDate’ column to a date type, it is already being read in that way (note the D in the col_types above). There is no data type conversion that you need to do with the Macatawa data at all (at least for this assignment)
- You will count sites listing them by site type, instead of counting by station and month as you did with the Erie data. When you get to this part of the code you should use the command
count(SiteType, Site) - Do not print out the shared data. The shared data file is too large, and you should not try to print it out to the screen or in your rendered document. In Tutorial 4 we could print out the shared data (the erie data frame) because it was relatively small. You can use dim to print out the dimensions of the shared data
- Use the following names for the clean data files and the figure that you produce:
| Data set or figure | folder | file name |
|---|---|---|
| metadata | clean_data | macatawa_meta.rds |
| shared | clean_data | macatawa.rds |
| taxonomy | clean_data | macatawa_tax.rds |
| sequencing depth histogram | outputs | mac2024_seq_depth-fwdb.pdf |
Option 1: Modify the Quarto document from Tutorial 5.5 and run it on the cluster
The Quarto document from tutorial 5.5 already has all of the code you need to complete the analysis. However, the code is not well-documented. If you choose this option, you need to provide the missing text. This includes
- A brief introduction for the analysis
- A brief explanation for each code block and any output that it produces
- Interpretation of the sequencing depth histogram
You should use RStudio on your local computer to modify the Quarto document (as described in tutorial 5.5). After you have edited the code, submit a slurm job to render your final .html report, while creating the clean data files and plot output.
Option 2: Write your own Quarto document following Tutorial 4 and run it on the cluster
Instead of adding explanatory text to tutorial 5.5 you may write your own version starting with Tutorial 4 as a template. Your Quarto document still needs to include the same explanatory text listed under Option 1. If you choose this option you must run your code on HPC. If you have a Quarto file on your local computer that you want to use for this assignment, upload that file to the code directory in your watershedMacatawa directory. Then use RStudio on your local computer to edit the document on HPC (as described in tutorial 5.5). After you have edited the code, submit an appropriate slurm job to render your final .html report, while creating the clean data files and plot output.
Interpretation
Include a discussion of your sequencing depth histogram immediately after the plot appears in your rendered Quarto document. Describe the distribution of sequencing depths for the watershed data. Estimate how much of the data you would lose (how many samples) by filtering the data if you needed each sample to have at least 10,000 reads.
Setting up a local watershedMacatawa project
Since we will be doing some of our computations on HPC and some of them locally (e.g., plots), we need to keep up to date versions of the code, clean data, etc on your local computer. We will work with matching project directory structures in both places.
If you have not already done so create a matching project directory structure on your local computer. That is, use RStudio to,create a project called watershedMacatawa in your local Projects directory (see Tutorial 2). Create the standard subdirectories within your local watershedMacatawa directory: raw_data, clean_data, code, and outputs. If you want to keep a local copy of your slurm scripts you can also create a slurm subdirectory within your code directory. However, you will only ever need these when you are working on HPC.
Making a local copy of your code, output, and clean data files
After you have finalized your code for this assignment, rendered your final .html report, and created the output and clean data files (should occur during rendering), you should download the updated files to your local computer. Copy your .qmd and .html files from the code directory on HPC to the code directory in your local watershedMacatawa directory. Also copy the clean data and output files (see the table above) from HPC to the corresponding local directories. You may also copy the raw data files to your local computer, but most of the local computations you will do in the future will use the clean data files only.
What you need to turn in
You need to turn in three files (upload to Moodle):
- Your Quarto (.qmd) file showing your interpretation, explanation, code, and appropriate output.
- Your rendered Quarto report (.html)
- Your sequencing depth histogram (.pdf) saved as a separate pdf file (using the ggsave command)