--- title: "Worksheet 5" output: pdf_document: default html_notebook: default --- ## Author: Enter name of the author here ## Discussants: Enter the names of other people who you discussed this problem set with $\\$ The purpose of this worksheet is to learn to use the dplyr functions to manipulate data and to review what we have covered so far (the dplyr functions will be useful when you engage with the class projet). To do so we will examine pitching statistics. Reviewing previous worksheets might be helpful for completing this worksheet. Some functions that will be useful for these exercises include: dim(), hist(), mean(), median(), sd(), sum(), quantile(), boxplot(), cor(), plot(), lm(), abline(), coef(), filter(), mutate(), group_by(), summarize(), arrange(). ```{r message=FALSE, warning=FALSE, tidy=TRUE, echo = FALSE} library(knitr) # makes sure the code is wrapped to fit when it creats a pdf opts_chunk$set(tidy.opts=list(width.cutoff=60)) set.seed(1) # set the random number generator to always give the same sequence of random numbers library('dplyr') # use the dplyr package to manipulate data frames library('Lahman') Pitching <- Pitching ``` $\\$ **Exercise 1:** In the following analyses we will examine some pitching statistics. Let's start by creating a data frame that has the pitching statistics from all pitchers who pitched at least 200 outs (i.e., pitchers who were pitching where IPouts >= 200 outs were recorded). To get the data from all pitchers, we use the 'Lahman' package which has been loaded using the library('Lahman') function in the initialization R chunk above. The Lahman package contains a data frame called Pitching that has pitching statistics from every pitcher (regardless of how many outs they pitched in a season). Use the dplyr filter() function on the Pitching data frame to create another data frame called pitchers.200outs that only has season data from pitchers who pitched for 200 outs. Note: the variable IPouts in the Pitching data frame lists how many outs a pitcher had in a season. Report how many cases there are in the pitchers.200outs data frame. The variable strike outs (SO) is the number of batters a pitcher was able to strike out in a season. Also plot a histogram of the number of strike outs that all players who pitched for 200 outs had and discribe the shape of the distribution. Is the mean or median higher for this distribution? ```{r message=FALSE, warning=FALSE, tidy=TRUE} ``` **Answers:** $\\$ **Exercise 2:** Now use the data in the data frame pitching.200outs you created above to come up with a number of how many stike outs a *good* pitcher would have (there is not one precise answer here, but just use the data to get a reasonable number and describe your rational for chosing this number). Also, determine whether the number of strike outs pitchers had in 1980 appears different from the number of strikes pitchers had in 2010. Hint: the filter() function, and the yearID variable will be useful, and also remember to use the two equal signs == inside the filter() function. Report what an impressive number of strike outs is for a pitcher and give evidence, through statistics and visualizations, about whether the number of strike outs pitchers recorded in 1980 appears similar to the number of strike outs pitchers recorded in the 2010's. ```{r message=FALSE, warning=FALSE} ``` **Answers:** $\\$ **Exercise 3:** The dplyr function mutate() can be used to add new variables to a data frame. The code below first gets all pitchers who pitched for 500 outs and then uses the mutate() function to add a variable called SO.rate that has the number of strike outs recorded per out pitched (SO/IPouts). Run this code and also add code that creates a variable called BB.rate that has the number of walks per out pitched. We can then examine whether there is a relationship between the percentage of batters a pitcher walks and the percentage of batters a pitcher strikes out. Create a scatter plot to see if there is a relationship between strike out rate and walk rate, and report the correlation between these rates. Finally create a linear regression model between these rates using the lm() function and add a regression line to the plot using the abline() function. Does there seem to be a relationship between a pitcher's strike out rate and their walk rate? Also report the slope of the regression line and state how much we would predict a pitcher's walk rate to increase if their strike out rate increased by .01. ```{r message=FALSE, warning=FALSE, tidy=TRUE} pitching.500outs <- filter(Pitching, IPouts > 500) pitching.500outs <- mutate(pitching.500outs, SO.rate = SO/IPouts) # first add code here to get the walk rate and then continue with the rest of the question... ``` **Answers:** $\\$ **Exercise 4:** Let's use multiple regression to build a model to predict a pitcher's strike out rate as a function of the walk rate, their earned run average (ERA), and the number of home runs they gave up (HR). To do this use the pitching.500outs that has the statistics from pitchers who pitched for at least 500 outs. Report what the root mean squared error (RMSE) is for making predictions on the data that was used to fit the model. Do you think the RMSE would be higher or lower if the model was trained on half the data and then predictions were made on the second half of the data? ```{r message=FALSE, warning=FALSE, tidy=TRUE} ``` **Answers:** $\\$ **Bonus question 1 (this question is optional):** As discussed in class, building a linear regression model and then calcuating the RMSE on the same data can lead to an unrealistic estimate of how good the linear model is at predicting new data points. A better way to estimate how good a linear regression model is at making predictions is to build the model on one set of data and then calculate the RMSE on a different set of data. The code below divides our data into two sets, a training set and a test set that can be used to get a more accurate estimate of the RMSE. Recreate the linear regression model for predicted a pitchers strike out rate as a function of their walk rate, their ERA and the number of home runs they gave up but this time *only use the training data* to build the model. Then make predictions and compute the RMSE *using the test data* to actually see what the crosss-valided RMSE is. Does the cross-validated RMSE appear similar to the RMSE you calculated in question 4? ```{r message=FALSE, warning=FALSE, tidy=TRUE} # Uncomment the code below to create training and test data sets # num.cases <- dim(pitching.500outs)[1] # rand.inds <- sample(1:num.cases) # half.point <- round(num.cases/2) # train.inds <- rand.inds[1:half.point] # test.inds <- rand.inds[(1 + half.point):num.cases] # train.data <- pitching.500outs[train.inds, ] # test.data <- pitching.500outs[test.inds, ] # bulid your linear model on the training data and get the RMSE on the test data here... ``` **Answers:** $\\$ **Bonus question 2 (this question is optional):** In class we also discussed the group_by() function which can be used to group data section of data together, and the summarize() function which can apply an function separately to each group. Use these functions on the original Pitching data frame to determine which baseball team has recorded the most strike outs over the total of their existance (hint: the arrange() function will be useful here too). ```{r message=FALSE, warning=FALSE, tidy=TRUE} ``` **Answers:** $\\$ **Bonus question 3: (this question is optional)** We can also use the filter() function to get data from an individual player. To do this we will need to merge two data sets together - one data set contains the players' names, and the other contains there pitching statistic. The code below uses the merge() function to combine these two datasets into a date set called merged.master.pitching. We can then get a player's individual data using the filter function specifying the first and last name of a player for the variables firstName and lastName. For example, the code below can be used to get the data from Madison Bumgarner. Another all time great Giant's pitcher is Christy Mathewson. Get Christy Mathewson's data can compare Christy and Madison's statistics for their 6th seasons. Who appears to be better during their 6th season relative to their peers? (hint: get all the data from each player's 6th season using filter(merged.master.pitching, yearID == ...) and proceed from there). ```{r message=FALSE, warning=FALSE, tidy=TRUE} merged.master.pitching <- merge(Master, Pitching) Bumgarner.data <- filter(merged.master.pitching, nameFirst == "Madison", nameLast == "Bumgarner") ``` **Answers:**