Description

Critical group 2: Avery Davidowitz, Josh Iden, Mathew Katz, Tyler Brown, Gabriel Santos, John Ledesm
knitr::opts_chunk$set(echo = FALSE, class.source = “codechunk”) library(mice) library(car) library(RCurl) library(ggplot2) library(reshape2) library(corrplot) library(reshape) library(mixtools) library(tidyverse) library(GGally) library(MASS) library(faraway) library(RCurl) library(tidyr) library(dplyr)
Problem Statement and Goals
Our objective is to make a linear regression model that can predict how many wins a baseball team will have in a season based on some metrics.
The variables we have been provided theoretically have positive or negative effects on the total number of wins.
The goal is to establish which variables are correlated with wins, as well as finding out if some of the variables can be consolidated using known conventional baseball-stats algorithms like SABER.
Data Exploration
Viewing Data
The data contains 17 columns. The index column will be ignored for analysis purposes, and so that leaves the other 16. TARGET_WINS is the variable we want to investigate with regards to how well it is correlated with the other columns. Every row represents a baseball team and its performance during a particular season. TARGET_WINS is the number of wins, and each column after that represents a particular metric for the season. For example, TEAM_PITCHING_E represents how many times an opposing team made a pitching mistake during the season. ’TEAM_BATTING_H‘ represents how many base hits by batters occurred for that team during the season. There are four categories of feature types:
• Batting
• Baserunning
• Pitching
• Fielding
## TARGET_WINS TEAM_BATTING_H TEAM_BATTING_2B TEAM_BATTING_3B
## Min. : 0.00 Min. : 891 Min. : 69.0 Min. : 0.00
## 1st Qu.: 71.00 1st Qu.:1383 1st Qu.:208.0 1st Qu.: 34.00
## Median : 82.00 Median :1454 Median :238.0 Median : 47.00
## Mean : 80.79 Mean :1469 Mean :241.2 Mean : 55.25
## 3rd Qu.: 92.00 3rd Qu.:1537 3rd Qu.:273.0 3rd Qu.: 72.00
## Max. :146.00 Max. :2554 Max. :458.0 Max. :223.00
##
## TEAM_BATTING_HR TEAM_BATTING_BB TEAM_BATTING_SO TEAM_BASERUN_SB
## Min. : 0.00 Min. : 0.0 Min. : 0.0 Min. : 0.0
## 1st Qu.: 42.00 1st Qu.:451.0 1st Qu.: 548.0 1st Qu.: 66.0
## Median :102.00 Median :512.0 Median : 750.0 Median :101.0
## Mean : 99.61 Mean :501.6 Mean : 735.6 Mean :124.8
## 3rd Qu.:147.00 3rd Qu.:580.0 3rd Qu.: 930.0 3rd Qu.:156.0
## Max. :264.00 Max. :878.0 Max. :1399.0 Max. :697.0
## NA’s :102 NA’s :131
## TEAM_BASERUN_CS TEAM_BATTING_HBP TEAM_PITCHING_H TEAM_PITCHING_HR
## Min. : 0.0 Min. :29.00 Min. : 1137 Min. : 0.0
## 1st Qu.: 38.0 1st Qu.:50.50 1st Qu.: 1419 1st Qu.: 50.0
## Median : 49.0 Median :58.00 Median : 1518 Median :107.0
## Mean : 52.8 Mean :59.36 Mean : 1779 Mean :105.7
## 3rd Qu.: 62.0 3rd Qu.:67.00 3rd Qu.: 1682 3rd Qu.:150.0
## Max. :201.0 Max. :95.00 Max. :30132 Max. :343.0
## NA’s :772 NA’s :2085
## TEAM_PITCHING_BB TEAM_PITCHING_SO TEAM_FIELDING_E TEAM_FIELDING_DP
## Min. : 0.0 Min. : 0.0 Min. : 65.0 Min. : 52.0
## 1st Qu.: 476.0 1st Qu.: 615.0 1st Qu.: 127.0 1st Qu.:131.0
## Median : 536.5 Median : 813.5 Median : 159.0 Median :149.0
## Mean : 553.0 Mean : 817.7 Mean : 246.5 Mean :146.4
## 3rd Qu.: 611.0 3rd Qu.: 968.0 3rd Qu.: 249.2 3rd Qu.:164.0
## Max. :3645.0 Max. :19278.0 Max. :1898.0 Max. :228.0
## NA’s :102 NA’s :286
According to the data, we can see that that target variable is roughly normally distributed, with a mean of total wins around 80 games.
It might make sense, as a standard season is 162 games, we would expect that the average number of wins would be roughly half of this value.
There are a few columns which appear to have outliers, particularly TEAM_PITCHING_H.
Exploration
Some of the columns have missing values.
It is possible that an entire season could be played without strikeouts by batters.
Number of missing values for each predictor

We were able to observe some atypical values (outliers), some values are too high, For example, a team with more than 20,000 strikeouts.
All variables

ind
From the chart above, we can see that the most problematic features include TEAM_PITCHING_H, TEAM_PITCHING_BB, and TEAM_PITCHING_SO. Where available we will employ cutoffs based on third party reference data such as baseball-almanac.com. If there is no available data, we will use other logical imputation methods to replace the outliers with reasonable values more fit to the data.
Data disribution
Histograms All Variables

We can see that some of the variables are skewed to the right or the left like TEAM_PITCHING_SO. Some of them even have more than one spike (bimodal) like TEAM_PITCHING_H. Our target feature is very well normally distributed.
Correlation
This is exploration of how the variables correlate with wins. In the chart below we can see that some of these variables correlate as we would expect with the number of wins – such as TEAM_BATTING correlating positively with wins. However some of them did not make sense- like TEAM_PITCHING_SO having a negative correlation with wins. We made this chart to get a general idea of how each variable related to the number of wins.
Correlation against the target variable for each predictor

Feature Multicollinearity
We want to know which characteristics are correlated, by means of a correlation graph.
TEAM_BATTING_H
TEAM_BATTING_2B
TEAM_BATTING_3B
TEAM_BATTING_HR
TEAM_BATTING_BB
TEAM_BATTING_SO
TEAM_BASERUN_SB
TEAM_BASERUN_CS
TEAM_BATTING_HBP
TEAM_PITCHING_H
TEAM_PITCHING_HR
TEAM_PITCHING_BB
TEAM_PITCHING_SO
TEAM_FIELDING_E
0.47
0.31 0.56
−0.12 0.21 0.04
0.42 0.40 0.25 −0.22
0.47 0.20 0.20 −0.21 0.46
−0.23 −0.34 −0.06 −0.19 0.21 0.22
0.01 0.07 −0.19 0.17 −0.19 −0.09 −0.07
−0.18 −0.09 −0.20 0.23 −0.28 −0.21 −0.06 0.62
0.07 −0.03 0.05 −0.17 0.11 0.05 0.22 −0.06 −0.07
0.47 1.00 0.56 0.21 0.40 0.20 −0.34 0.07 −0.09 −0.03
0.42 0.39 0.25 −0.22 1.00 0.46 0.21 −0.19 −0.28 0.11 0.39
0.47 0.20 0.20 −0.21 0.46 1.00 0.22 −0.09 −0.21 0.05 0.20 0.46
−0.23 −0.34 −0.07 −0.19 0.21 0.22 1.00 −0.07 −0.06 0.22 −0.34 0.21 0.22
−0.39 −0.25 −0.19 −0.07 0.02 −0.08 0.31 0.04 0.21 0.04 −0.25 0.02 −0.08 0.31
−0.20 0.02 −0.02 0.13 −0.06 −0.08 −0.12 −0.13 −0.01 −0.07 0.01 −0.06 −0.08 −0.12 0.04
TEAM_FIELDING_DP

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1
we notice that there are a few features which exhibit very strong positive correlation. In particular:
• TEAM_PITCHING_H & TEAM_BATTIING_H == 1.0 correlation
• TEAM_PITCHING_HR & TEAM_BATTING_HR == 1.0 correlation • TEAM_PITCHING_BB & TEAM_BATTING_BB == 1.0 correlation
• TEAM_PITCHING_SO & TEAM_BATTING_SO == 1.0 correlation
However, we must consider that these initial correlation values could be influenced by the fact that missing values and outliers have yet to be addressed.
Data Preparation
Renaming Column Names
Keeping column names short and readable is important in order to practice “table hygiene”. Therefore, new column names were generated and are shown on Table XX.
Original Column Name New Column Name
TARGET_WINS target TEAM_BATTING_H bat_h TEAM_BATTING_2B bat_2b
TEAM_BATTING_3B bat_3b TEAM_BATTING_HR bat_hr TEAM_BATTING_BB bat_bb
TEAM_BATTING_HBP bat_hbp
Original Column Name New Column Name
TEAM_BATTING_SO bat_so
TEAM_BASERUN_CS bas_cs
TEAM_FIELDING_E f_e
TEAM_FIELDING_DP f_dp
TEAM_PITCHING_BB p_bb
TEAM_PITCHING_H p_h
TEAM_PITCHING_HR p_hr
TEAM_PITCHING_SO p_so
Dealing with Missing Values
There are 6 features that have missing values:
• Strikeouts by batters (5%): Should use median or regression model for imputation
• Stolen bases (6%): Stolen bases weren’t tracked officially until 1887, which means some of the missing data could be from 1871-1886. These values could be imputed.
• Caught stealing (34%): Stolen bases weren’t tracked officially until 1887, so some of the missing data could be from 1871-1886. These values could be imputed.
• Batter hit by pitch (92%): This predictor will be removed from the analysis as too many of its values are missing.
• Strikeouts by pitchers (4%): Should use median or regression model for imputation
• Double plays (12%): Should use median or regression model for imputation
Tabachnick and Fidell
Our team decided to employ another technique to handle the missing values: Multiple Regression Imputation using the MICE package.
The MICE package in R implements a methodology where each incomplete variable is imputed by a separate model. Alice points out that plausible values are drawn from a distribution specifically designed for each missing datapoint. Many imputation methods can be used within the package. The one that was selected for the data being analyzed in this report is PMM (Predictive Mean Matching), which is used for quantitative data.
Van Buuren explains that PMM works by selecting values from the observed/already existing data that would most likely belong to the variable in the observation with the missing value. The advantage of this is that it selects values that must exist from the observed data, so no negative values will be used to impute missing data.Not only that, it circumvents the shrinking of errors by using multiple regression models. The variability between the different imputed values gives a wider, but more correct standard error. Uncertainty is inherent in imputation which is why having multiple imputed values is important. Not only that. Marshall et al. 2010 points out that:
Removal of bat_hbp- NA

Following use of the MICE package, we can visualize the distributions of the imputed versus existing data points as shown on Figure 6. The density of the imputed data for each imputed dataset is shown in magenta. The density of the observed data is shown in blue. For the MICE algorithm, the number of multiple imputations was set to five. The imputed distribution for bas_sb and p_so look close to the original data distribution which is good. The imputed data distributions for the other variables do not match so closely to the original data. Reasons include:
• Some of the variables are bimodal in nature (which is why in bas_cs for example, there is bimodality in the imputed distributions).
• 34% of the data for bas_cs is missing, which is above 5%, while the missing data for p_so only makes up 4% of the total amount of missing data for that predictor.
• 12% of the data for f_dp is missing, which is above 5%, while the missing data for p_so only makes up 4% of the total amount of missing data for that predictor.
Analysis of Outliers
Several predictors contained outliers that contradicted with existing baseball statistics or fell out of an “acceptable” range given the feature’s inherent distribution. These features are:
• bat_h: The most hits by team in a season is 1783. Therefore, any values above 1,783 were replaced with the median for the predictor (Source).
• p_h: We could not find any suitable statistics from outside sources for this feature. However, we can apply interquartile outlier analysis. By analyzing a given feature, those datapoints which fall above or below an “acceptable” range can be identified given the features inherent distribution.
• p_so: The record for most strikeouts in a season is 1595. Anything above this should be removed or imputed (Source).
• f_e: The record for most errors in a season is 886. Anything above this should be removed or imputed
(Source).
• p_bb: We could not find any suitable statistics from outside sources for this feature. However, we can apply interquartile outlier analysis. By analyzing a given feature, those datapoints which fall above or below an “acceptable” range can be identified given the features inherent distribution.
After replacing the above outliers, we can visualize the improved distributions by use of a boxplot.
Distributions after outlier analysis and imputing NA Values

While there are still outliers present in the dataset, particularly for bas_sb and f_e, we can see a large improvement from before. All features are wihin the range 0-2500. We can attempt to further deal with outliers should the need arise, but for now we will accept this distribution.
Box-Cox Transformation for skewed variables
Based on the previous distribution plot (using histograms) we noticed that a select group of columns exhibited non-normal skew. In particular, the following columns showed signs of left-skew:
Skewed variables

In order to address this skewness and attempt to normalize these features for future modeling, we will employ box-cox transformations. Because some of these values include 0, we will need to replace any zero values with infintesimmaly small, non-zero values.
The λ’s that were used to transform the skewed variables are shown on Table 2.
Histograms for transformed variables

λ’s for skewed variables
Column Name λ
bat_3b 0.400
bas_sb 0.220
bas_cs 0.232
f_e -0.753
p_bb 0.460
p_h -2.034
As we can see from the above, the boxcox transformations on the selected features performed extremely well. We can see that all features included now exhibit normal or near-normal distributions around their respective centers.
Dealing with Bimodal Variables
Bimodal distributions in data are interesting, in that they represent features which actually contain multiple (2) inherent systems resulting in separated distributional peaks. Figure XX shows that bimodality is present in bat_so, p_hr, bat_hr. While a Box-Cox transformation could have been undertaken in order to transform the bimodal variables to a normal distribution. However, this throws away important information that is inherent in the bimodal variable itself. The fact that the variable is bimodal in the first place is essentially ignored, and the predicted values in the linear multiple regression model will not reflect this bimodality.
Our approach to solving this is to create dummy variables representing which side of the local minimum each data point falls with respect to it’s original bimodal distribution. First, two histograms were fit to these variables using the mixtools package. Then, the intersection point between the two histograms was determined by solving for c. Where
µ2σ12− σ2(µ1σ2 + σ1q(µ1− µ2)2 + 2(σ12− σ22)logσσ12)
c = σ12− σ22
Where µ1 and σ1 are the mean and standard deviation for the left distribution and ν2 and σ2 are the mean and standard deviation for the right distribution.
Finds where two histograms intersect and Creates dummy variables based on bimodal data:
Density curves for each bimodal predictor with two normal distributions fit to each peak.
Density Curves

0 200 400 600 800 1000 1200 1400
Data
Density Curves

0 50 100 150 200 250 300 350
Data
Density Curves

0 50 100 150 200 250
Data
Summary of bimodal dummy variable generation
Column
Name µ1 µ2 σ1 σ2 c Count of 0’s
bat_so 606.31 972.61 199.88 114.06 806.38 969
p_hr 31.43 127.37 14.39 52.08 60.93 1602
bat_hr 26.55 125.06 13.10 48.72 54.93 1583
Bar graphs for each of the bimodal dummy variables.
0 represents the amount of observations for the original variable where the value was above c, while 1 represents the amount of observations below c*
0 1 0 1
bi_bat_so bi_p_hr bi_bat_hr
Saber Model
We would like to employ outside analysis in order to engineer new, potentially powerful features. Popularized in the movie “Money ball”, the SABERMETRICS model for baseball analysis includes a feature known as BsR (base runs). This statistic estimates the amount of runs a team should score.
Base Runs (BsR) is a run estimator developed by David Smyth in the early 1990s. It was inspired by Bill James’ Runs Created, and like Runs Created it is a dynamic model of team run scoring. Many different versions of Base Runs have been introduced by several sabermetricians, to accommodate different data sets and philosophies. However, all Base Runs formulas take the form:
A*B/(B + C) + D
BSR = AB/(B + A) + C
where:
A = TEAM_BATTING_1B+TEAM_BATTING_2B+TEAM_BATTING_3B+TEAM_BATTING_BB
B = 1.02(1.4TEAM_TOTAL_BASES − 0.6TEAM_BATTING_H + 0.1TEAM_BATTING_BB)
C = TEAM_BATTING_HR
Histogram of BSR Predictor

BSR
Reviewing the correlations
After performing multiple cleaning and imputation steps, we would like to visualize again the correlations between features and their target, as well as between features themselves.

0.35
0.29 0.57
0.18 0.09 0.44
0.23 0.09 0.26 0.51
−0.04 −0.31 0.18 0.72 0.38
0.19 0.12 0.45 0.97 0.46 0.66
−0.07 −0.28 0.13 0.55 0.16 0.87 0.57
−0.03 0.08 0.31 0.51 0.40 0.29 0.49 0.14
0.13 0.32 −0.13 −0.65 −0.28 −0.68 −0.59 −0.53 −0.41
0.13 0.04 −0.18 −0.46 −0.26 −0.25 −0.42 −0.09 −0.57 0.46
0.02 0.01 −0.25 −0.52 −0.23 −0.32 −0.50 −0.24 −0.41 0.45 0.54
−0.09 0.07 −0.36 −0.75 −0.37 −0.64 −0.72 −0.45 −0.56 0.71 0.53 0.48
0.20 0.16 0.21 0.29 0.72 0.09 0.32 0.04 0.22 −0.11 −0.08 −0.16 −0.19
0.18 0.69 0.30 −0.18 −0.02 −0.46 −0.12 −0.35 −0.10 0.47 0.18 0.12 0.34 0.16
0.14 0.70 −0.06 −0.58 −0.27 −0.73 −0.54 −0.60 −0.30 0.69 0.33 0.36 0.59 −0.07 0.64
0.34 0.63 0.75 0.82 0.45 0.39 0.82 0.28 0.45 −0.32 −0.33 −0.40 −0.55 0.32 0.25 −0.09
0.32 0.55 0.68 0.88 0.49 0.46 0.87 0.33 0.49 −0.39 −0.37 −0.44 −0.61 0.33 0.17 −0.17 0.99
bat_h bat_2b bat_hr bat_bb bat_so p_hr p_so f_dp bat_3b bas_sb bas_cs f_e p_bb p_h bat_1b total_bases saber

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1
These correlation values make much more sense than before. We can see that features no longer have 1.0 correlations, which in general are highly unlikely to occur naturally. The new most correlated (and least correlated) features are as follows:
• p_hr & bat_hr (0.97): This is an interested correlation, as we would not have initially expected the amount of home runs allowed to be correlated with the number of home runs achieved from a team. However, one could make the argument that a team which focuses on offense would similarly be lacking in defense.
• bat_1b & bat_so (-0.73): These features are negatively correlated, which makes intuitive sense. If a team has many players making it to base, then conversely we would expect that this team would have less strikeouts at bat.
• bat_so & p_so (0.87): These features intuitively should not have such high correlation. Similar to above, we would not expect the performance of batter strikeouts to have any relationship to the performance of pitching strikeouts on the same team.
Target correlation plot after cleaning

After applying all transformations and imputations, we can see that the feature correlation with the target variable has also improved. Features predicted to have positive correlations (as provided by the assignment guide) do tend to have positive correlations. Similarly, features with expected negative correlations behave as described. This provides us some level of validation as we take the next steps with model building.
Build Models
Create data frame with 0 rows and 3 columns and Create function to update the tracker
Examine base model, no transformations, no engineering
Our first model (Base model) will use all of the initially provided columns, after cleaning and imputation. We will use the results of this model to understand a baseline for our future model development.
##
## Coefficients: (1 not defined because of singularities)
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.9253e+02 7.4378e+01 9.3110 < 2.2e-16
## bat_h 1.8255e-02 3.8878e-03 4.6955 2.819e-06
## bat_2b 2.7989e-02 8.6740e-03 3.2268 0.0012697
## bat_hr 7.1466e-02 2.8495e-02 2.5080 0.0122103
## bat_bb 3.9873e-02 4.5933e-03 8.6807 < 2.2e-16
## bat_so -1.9114e-02 4.1948e-03 -4.5566 5.476e-06
## p_hr 1.5501e-02 2.4562e-02 0.6311 0.5280260
## p_bb -5.1417e-01 1.4920e-01 -3.4461 0.0005791
## p_so 1.4940e-03 3.2302e-03 0.4625 0.6437681
## f_dp -1.0332e-01 1.3232e-02 -7.8084 8.784e-15
## bat_3b 1.4218e+00 2.0473e-01 6.9449 4.930e-12
## bas_sb 1.8485e+00 2.0479e-01 9.0267 < 2.2e-16
## bas_cs 1.3926e-01 2.5226e-01 0.5520 0.5809747
## f_e -5.0564e+02 5.6774e+01 -8.9062 < 2.2e-16
##
## n = 2276, p = 14, Residual SE = 13.37938, R-Squared = 0.28
Based on the above output, we can see that this model performs relateively poorly against the training data. However, as this is our base model, we will assess the performance of all future models against this value. Moving forward, if we can lift the Adjusted rˆ2 to above 0.3, we will consider it a general improvement.
Evaluate SABER model
The next model we would like to evaluate is the SABER model. Here we will use all original features, and additionally we will include the engineered SABER metrics. Hopefully we will see a lift in performance after utilizing these industry-derived features.
##
## Coefficients: (3 not defined because of singularities)
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.8848e+02 7.4296e+01 9.2667 < 2.2e-16
## bat_h -2.1195e-01 5.0614e-02 -4.1875 2.928e-05
## bat_2b -1.0044e-01 2.9629e-02 -3.3899 0.0007111
## bat_hr -7.1592e-01 1.7701e-01 -4.0446 5.416e-05
## bat_bb 2.5793e-02 5.7484e-03 4.4870 7.586e-06
## bat_so -2.5209e-02 4.4272e-03 -5.6942 1.401e-08
## p_hr -2.3380e-02 2.5598e-02 -0.9134 0.3611534
## p_so 4.6712e-03 3.2620e-03 1.4320 0.1522772
## f_dp -1.1571e-01 1.3448e-02 -8.6043 < 2.2e-16
## bat_3b 1.0546e+00 2.1917e-01 4.8115 1.597e-06
## bas_sb 1.9034e+00 2.0605e-01 9.2376 < 2.2e-16
## bas_cs 2.5214e-01 2.5180e-01 1.0014 0.3167610
## f_e -5.0468e+02 5.6876e+01 -8.8733 < 2.2e-16
## p_bb -3.8029e-01 1.5221e-01 -2.4984 0.0125459
## bi_bat_so1 -1.7174e+00 1.0639e+00 -1.6143 0.1065933
## bi_p_hr1 -3.4765e+00 1.9107e+00 -1.8195 0.0689662
## bi_bat_hr1 5.8917e+00 2.0429e+00 2.8839 0.0039649
## saber 4.8640e-01 1.0613e-01 4.5829 4.837e-06
##
## n = 2276, p = 18, Residual SE = 13.28237, R-Squared = 0.29
As expected, we did see a lift in perfomance after including SABER metrics. However, the lift was hardly significant. We are still below 0.3 Adjusted Rˆ2.
SABER reduced
Here we will test out a more parsimonious version of the above SABER model. In the spirit of simplifying the model for human use and understanding, we will select only the features that have high significance from the above SABER model. Additionally, we will exlude any features which were included as part of the construction of SABER, in order to reduce inherent multicollinearity.
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.8049e+02 6.8429e+01 7.0217 2.884e-12
## saber 5.2232e-02 3.1766e-03 16.4425 < 2.2e-16
## bi_bat_hr1 3.0012e-02 1.0873e+00 0.0276 0.978
## f_e -3.4661e+02 5.1743e+01 -6.6987 2.643e-11
## bas_sb 2.1296e+00 1.9113e-01 11.1425 < 2.2e-16
## f_dp -1.0406e-01 1.3536e-02 -7.6882 2.208e-14
## bat_so -2.0973e-02 1.6315e-03 -12.8553 < 2.2e-16
## bat_bb 2.7673e-02 2.8093e-03 9.8504 < 2.2e-16
##
## n = 2276, p = 8, Residual SE = 13.56093, R-Squared = 0.26
While the Adjusted Rˆ2 has been slightly reduced to 0.26, we have also significantly reduced the complexity of the model. This provides value in itself, as the model can be more easily distributed to players and coaches.
Step AIC
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.8681e+02 7.2984e+01 9.4104 < 2.2e-16
## bat_h 1.8176e-02 3.8290e-03 4.7469 2.195e-06
## bat_2b 2.8459e-02 8.5675e-03 3.3218 0.0009086
## bat_hr 8.6889e-02 9.9994e-03 8.6895 < 2.2e-16
## bat_bb 3.7782e-02 3.9872e-03 9.4760 < 2.2e-16
## bat_so -1.7546e-02 2.2134e-03 -7.9270 3.491e-15
## p_bb -4.5775e-01 1.3582e-01 -3.3703 0.0007635
## f_dp -1.0287e-01 1.3187e-02 -7.8004 9.342e-15
## bat_3b 1.4629e+00 2.0035e-01 7.3020 3.905e-13
## bas_sb 1.8969e+00 1.9331e-01 9.8128 < 2.2e-16
## f_e -5.0186e+02 5.5816e+01 -8.9913 < 2.2e-16
##
## n = 2276, p = 11, Residual SE = 13.37488, R-Squared = 0.28
Square Root Step AIC
The following model was generated using the same AIC methodology, except that the target variable was square rooted.
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.2631e+01 4.2723e+00 9.9785 < 2.2e-16
## bat_h -1.2958e-02 2.9130e-03 -4.4484 9.072e-06
## bat_2b -6.5019e-03 1.7158e-03 -3.7894 0.0001550
## bat_hr -4.6426e-02 1.0289e-02 -4.5123 6.741e-06
## bat_bb 1.7533e-03 3.1617e-04 5.5454 3.275e-08
## bat_so -9.7840e-04 1.5165e-04 -6.4517 1.349e-10
## f_dp -6.4855e-03 7.8615e-04 -8.2498 2.661e-16
## bat_3b 6.3764e-02 1.2722e-02 5.0123 5.796e-07
## bas_sb 1.1348e-01 1.2044e-02 9.4223 < 2.2e-16
## bas_cs 2.7857e-02 1.4665e-02 1.8996 0.0576168
## f_e -2.8383e+01 3.2744e+00 -8.6683 < 2.2e-16
## p_bb -3.0901e-02 8.2592e-03 -3.7414 0.0001876
## bi_bat_so1 -9.6903e-02 6.2247e-02 -1.5567 0.1196707
## bi_p_hr1 -1.8528e-01 1.0787e-01 -1.7176 0.0860074
## bi_bat_hr1 3.1635e-01 1.1674e-01 2.7099 0.0067814
## saber 3.0081e-02 6.1195e-03 4.9156 9.484e-07
##
## n = 2276, p = 16, Residual SE = 0.77728, R-Squared = 0.31
Model Selection

0.303144087087744
0.28899695594346
0.279058928565008
0.278573831105552
0.258861608098488
Base Model Saber Model Saber Reduced Step AIC Step AIC Sqrt model
The model that is ultimately chosen for this analysis is Step AIC Square Root. We were able to increase over the base model by 3%. AIC is a measure of multicollinearity so the selection process parsed out variables that were highly colinear with other variables, giving us a model that has the lowest AIC values based on a select number of predictors. This is important because this model needs to be used and understood by professionals in the industry; the step AIC model ensures that only the most prominent features are included.

The QQ plot shows that the data is centered in the middle, but there is significant amount of residuals in the middle of the distribution. This is known as the “thin tail” phenomenon. Normal distributions with “thin tails” correspond to the first quantiles occurring at larger than expected values and the last quantiles occurring at less than expected values. Notice that the “thin tailed” Q-Q plot is a reflection of a “fat tailed” Q-Q plot, which is the opposite phenomenon, across the X-Y diagonal. The Residuals vs. Fitted, and scale-location plots show that the residuals are mostly centered around the zero line. However there is some skewness as a result of outliers that are marked numerically on the plot itself. The Residuals vs. Leverage plot shows that there are no residual values that exceed Cook’s distance, which is good.
Important Metrics for Step AIC Square Root Model Important metrics for the Step AIC Square Root model are:
• R-squared: 0.3027
• F-statistic: 71.54
• RSS: 1366.88 • MSE: 0.6
• RMSE: 0.774
Predictions on Evaluation Set
The predictions were generated using the evaluation set on the Square Root Step AIC model. These predictions are provided in the predictions.csv file.
Appendix
The R code that was used in the analyses shown in this report can be viewed here:
“RMD code”.
“Rpubs”.
• Thank you.