## Description

Critical group 2: Avery Davidowitz, Josh Iden, Mathew Katz, Tyler Brown, Gabriel Santos, John Ledesm

knitr::opts_chunk$set(echo = FALSE, class.source = “codechunk”) library(mice) library(car) library(RCurl) library(ggplot2) library(reshape2) library(corrplot) library(reshape) library(mixtools) library(tidyverse) library(GGally) library(MASS) library(faraway) library(RCurl) library(tidyr) library(dplyr)

Problem Statement and Goals

Our objective is to make a linear regression model that can predict how many wins a baseball team will have in a season based on some metrics.

The variables we have been provided theoretically have positive or negative effects on the total number of wins.

The goal is to establish which variables are correlated with wins, as well as finding out if some of the variables can be consolidated using known conventional baseball-stats algorithms like SABER.

Data Exploration

Viewing Data

The data contains 17 columns. The index column will be ignored for analysis purposes, and so that leaves the other 16. TARGET_WINS is the variable we want to investigate with regards to how well it is correlated with the other columns. Every row represents a baseball team and its performance during a particular season. TARGET_WINS is the number of wins, and each column after that represents a particular metric for the season. For example, TEAM_PITCHING_E represents how many times an opposing team made a pitching mistake during the season. ’TEAM_BATTING_H‘ represents how many base hits by batters occurred for that team during the season. There are four categories of feature types:

• Batting

• Baserunning

• Pitching

• Fielding

## TARGET_WINS TEAM_BATTING_H TEAM_BATTING_2B TEAM_BATTING_3B

## Min. : 0.00 Min. : 891 Min. : 69.0 Min. : 0.00

## 1st Qu.: 71.00 1st Qu.:1383 1st Qu.:208.0 1st Qu.: 34.00

## Median : 82.00 Median :1454 Median :238.0 Median : 47.00

## Mean : 80.79 Mean :1469 Mean :241.2 Mean : 55.25

## 3rd Qu.: 92.00 3rd Qu.:1537 3rd Qu.:273.0 3rd Qu.: 72.00

## Max. :146.00 Max. :2554 Max. :458.0 Max. :223.00

##

## TEAM_BATTING_HR TEAM_BATTING_BB TEAM_BATTING_SO TEAM_BASERUN_SB

## Min. : 0.00 Min. : 0.0 Min. : 0.0 Min. : 0.0

## 1st Qu.: 42.00 1st Qu.:451.0 1st Qu.: 548.0 1st Qu.: 66.0

## Median :102.00 Median :512.0 Median : 750.0 Median :101.0

## Mean : 99.61 Mean :501.6 Mean : 735.6 Mean :124.8

## 3rd Qu.:147.00 3rd Qu.:580.0 3rd Qu.: 930.0 3rd Qu.:156.0

## Max. :264.00 Max. :878.0 Max. :1399.0 Max. :697.0

## NA’s :102 NA’s :131

## TEAM_BASERUN_CS TEAM_BATTING_HBP TEAM_PITCHING_H TEAM_PITCHING_HR

## Min. : 0.0 Min. :29.00 Min. : 1137 Min. : 0.0

## 1st Qu.: 38.0 1st Qu.:50.50 1st Qu.: 1419 1st Qu.: 50.0

## Median : 49.0 Median :58.00 Median : 1518 Median :107.0

## Mean : 52.8 Mean :59.36 Mean : 1779 Mean :105.7

## 3rd Qu.: 62.0 3rd Qu.:67.00 3rd Qu.: 1682 3rd Qu.:150.0

## Max. :201.0 Max. :95.00 Max. :30132 Max. :343.0

## NA’s :772 NA’s :2085

## TEAM_PITCHING_BB TEAM_PITCHING_SO TEAM_FIELDING_E TEAM_FIELDING_DP

## Min. : 0.0 Min. : 0.0 Min. : 65.0 Min. : 52.0

## 1st Qu.: 476.0 1st Qu.: 615.0 1st Qu.: 127.0 1st Qu.:131.0

## Median : 536.5 Median : 813.5 Median : 159.0 Median :149.0

## Mean : 553.0 Mean : 817.7 Mean : 246.5 Mean :146.4

## 3rd Qu.: 611.0 3rd Qu.: 968.0 3rd Qu.: 249.2 3rd Qu.:164.0

## Max. :3645.0 Max. :19278.0 Max. :1898.0 Max. :228.0

## NA’s :102 NA’s :286

According to the data, we can see that that target variable is roughly normally distributed, with a mean of total wins around 80 games.

It might make sense, as a standard season is 162 games, we would expect that the average number of wins would be roughly half of this value.

There are a few columns which appear to have outliers, particularly TEAM_PITCHING_H.

Exploration

Some of the columns have missing values.

It is possible that an entire season could be played without strikeouts by batters.

Number of missing values for each predictor

We were able to observe some atypical values (outliers), some values are too high, For example, a team with more than 20,000 strikeouts.

All variables

ind

From the chart above, we can see that the most problematic features include TEAM_PITCHING_H, TEAM_PITCHING_BB, and TEAM_PITCHING_SO. Where available we will employ cutoffs based on third party reference data such as baseball-almanac.com. If there is no available data, we will use other logical imputation methods to replace the outliers with reasonable values more fit to the data.

Data disribution

Histograms All Variables

We can see that some of the variables are skewed to the right or the left like TEAM_PITCHING_SO. Some of them even have more than one spike (bimodal) like TEAM_PITCHING_H. Our target feature is very well normally distributed.

Correlation

This is exploration of how the variables correlate with wins. In the chart below we can see that some of these variables correlate as we would expect with the number of wins – such as TEAM_BATTING correlating positively with wins. However some of them did not make sense- like TEAM_PITCHING_SO having a negative correlation with wins. We made this chart to get a general idea of how each variable related to the number of wins.

Correlation against the target variable for each predictor

Feature Multicollinearity

We want to know which characteristics are correlated, by means of a correlation graph.

TEAM_BATTING_H

TEAM_BATTING_2B

TEAM_BATTING_3B

TEAM_BATTING_HR

TEAM_BATTING_BB

TEAM_BATTING_SO

TEAM_BASERUN_SB

TEAM_BASERUN_CS

TEAM_BATTING_HBP

TEAM_PITCHING_H

TEAM_PITCHING_HR

TEAM_PITCHING_BB

TEAM_PITCHING_SO

TEAM_FIELDING_E

0.47

0.31 0.56

−0.12 0.21 0.04

0.42 0.40 0.25 −0.22

0.47 0.20 0.20 −0.21 0.46

−0.23 −0.34 −0.06 −0.19 0.21 0.22

0.01 0.07 −0.19 0.17 −0.19 −0.09 −0.07

−0.18 −0.09 −0.20 0.23 −0.28 −0.21 −0.06 0.62

0.07 −0.03 0.05 −0.17 0.11 0.05 0.22 −0.06 −0.07

0.47 1.00 0.56 0.21 0.40 0.20 −0.34 0.07 −0.09 −0.03

0.42 0.39 0.25 −0.22 1.00 0.46 0.21 −0.19 −0.28 0.11 0.39

0.47 0.20 0.20 −0.21 0.46 1.00 0.22 −0.09 −0.21 0.05 0.20 0.46

−0.23 −0.34 −0.07 −0.19 0.21 0.22 1.00 −0.07 −0.06 0.22 −0.34 0.21 0.22

−0.39 −0.25 −0.19 −0.07 0.02 −0.08 0.31 0.04 0.21 0.04 −0.25 0.02 −0.08 0.31

−0.20 0.02 −0.02 0.13 −0.06 −0.08 −0.12 −0.13 −0.01 −0.07 0.01 −0.06 −0.08 −0.12 0.04

TEAM_FIELDING_DP

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1

we notice that there are a few features which exhibit very strong positive correlation. In particular:

• TEAM_PITCHING_H & TEAM_BATTIING_H == 1.0 correlation

• TEAM_PITCHING_HR & TEAM_BATTING_HR == 1.0 correlation • TEAM_PITCHING_BB & TEAM_BATTING_BB == 1.0 correlation

• TEAM_PITCHING_SO & TEAM_BATTING_SO == 1.0 correlation

However, we must consider that these initial correlation values could be influenced by the fact that missing values and outliers have yet to be addressed.

Data Preparation

Renaming Column Names

Keeping column names short and readable is important in order to practice “table hygiene”. Therefore, new column names were generated and are shown on Table XX.

Original Column Name New Column Name

TARGET_WINS target TEAM_BATTING_H bat_h TEAM_BATTING_2B bat_2b

TEAM_BATTING_3B bat_3b TEAM_BATTING_HR bat_hr TEAM_BATTING_BB bat_bb

TEAM_BATTING_HBP bat_hbp

Original Column Name New Column Name

TEAM_BATTING_SO bat_so

TEAM_BASERUN_CS bas_cs

TEAM_FIELDING_E f_e

TEAM_FIELDING_DP f_dp

TEAM_PITCHING_BB p_bb

TEAM_PITCHING_H p_h

TEAM_PITCHING_HR p_hr

TEAM_PITCHING_SO p_so

Dealing with Missing Values

There are 6 features that have missing values:

• Strikeouts by batters (5%): Should use median or regression model for imputation

• Stolen bases (6%): Stolen bases weren’t tracked officially until 1887, which means some of the missing data could be from 1871-1886. These values could be imputed.

• Caught stealing (34%): Stolen bases weren’t tracked officially until 1887, so some of the missing data could be from 1871-1886. These values could be imputed.

• Batter hit by pitch (92%): This predictor will be removed from the analysis as too many of its values are missing.

• Strikeouts by pitchers (4%): Should use median or regression model for imputation

• Double plays (12%): Should use median or regression model for imputation

Tabachnick and Fidell

Our team decided to employ another technique to handle the missing values: Multiple Regression Imputation using the MICE package.

The MICE package in R implements a methodology where each incomplete variable is imputed by a separate model. Alice points out that plausible values are drawn from a distribution specifically designed for each missing datapoint. Many imputation methods can be used within the package. The one that was selected for the data being analyzed in this report is PMM (Predictive Mean Matching), which is used for quantitative data.

Van Buuren explains that PMM works by selecting values from the observed/already existing data that would most likely belong to the variable in the observation with the missing value. The advantage of this is that it selects values that must exist from the observed data, so no negative values will be used to impute missing data.Not only that, it circumvents the shrinking of errors by using multiple regression models. The variability between the different imputed values gives a wider, but more correct standard error. Uncertainty is inherent in imputation which is why having multiple imputed values is important. Not only that. Marshall et al. 2010 points out that:

Removal of bat_hbp- NA

Following use of the MICE package, we can visualize the distributions of the imputed versus existing data points as shown on Figure 6. The density of the imputed data for each imputed dataset is shown in magenta. The density of the observed data is shown in blue. For the MICE algorithm, the number of multiple imputations was set to five. The imputed distribution for bas_sb and p_so look close to the original data distribution which is good. The imputed data distributions for the other variables do not match so closely to the original data. Reasons include:

• Some of the variables are bimodal in nature (which is why in bas_cs for example, there is bimodality in the imputed distributions).

• 34% of the data for bas_cs is missing, which is above 5%, while the missing data for p_so only makes up 4% of the total amount of missing data for that predictor.

• 12% of the data for f_dp is missing, which is above 5%, while the missing data for p_so only makes up 4% of the total amount of missing data for that predictor.

Analysis of Outliers

Several predictors contained outliers that contradicted with existing baseball statistics or fell out of an “acceptable” range given the feature’s inherent distribution. These features are:

• bat_h: The most hits by team in a season is 1783. Therefore, any values above 1,783 were replaced with the median for the predictor (Source).

• p_h: We could not find any suitable statistics from outside sources for this feature. However, we can apply interquartile outlier analysis. By analyzing a given feature, those datapoints which fall above or below an “acceptable” range can be identified given the features inherent distribution.

• p_so: The record for most strikeouts in a season is 1595. Anything above this should be removed or imputed (Source).

• f_e: The record for most errors in a season is 886. Anything above this should be removed or imputed

(Source).

• p_bb: We could not find any suitable statistics from outside sources for this feature. However, we can apply interquartile outlier analysis. By analyzing a given feature, those datapoints which fall above or below an “acceptable” range can be identified given the features inherent distribution.

After replacing the above outliers, we can visualize the improved distributions by use of a boxplot.

Distributions after outlier analysis and imputing NA Values

While there are still outliers present in the dataset, particularly for bas_sb and f_e, we can see a large improvement from before. All features are wihin the range 0-2500. We can attempt to further deal with outliers should the need arise, but for now we will accept this distribution.

Box-Cox Transformation for skewed variables

Based on the previous distribution plot (using histograms) we noticed that a select group of columns exhibited non-normal skew. In particular, the following columns showed signs of left-skew:

Skewed variables

In order to address this skewness and attempt to normalize these features for future modeling, we will employ box-cox transformations. Because some of these values include 0, we will need to replace any zero values with infintesimmaly small, non-zero values.

The λ’s that were used to transform the skewed variables are shown on Table 2.

Histograms for transformed variables

λ’s for skewed variables

Column Name λ

bat_3b 0.400

bas_sb 0.220

bas_cs 0.232

f_e -0.753

p_bb 0.460

p_h -2.034

As we can see from the above, the boxcox transformations on the selected features performed extremely well. We can see that all features included now exhibit normal or near-normal distributions around their respective centers.

Dealing with Bimodal Variables

Bimodal distributions in data are interesting, in that they represent features which actually contain multiple (2) inherent systems resulting in separated distributional peaks. Figure XX shows that bimodality is present in bat_so, p_hr, bat_hr. While a Box-Cox transformation could have been undertaken in order to transform the bimodal variables to a normal distribution. However, this throws away important information that is inherent in the bimodal variable itself. The fact that the variable is bimodal in the first place is essentially ignored, and the predicted values in the linear multiple regression model will not reflect this bimodality.

Our approach to solving this is to create dummy variables representing which side of the local minimum each data point falls with respect to it’s original bimodal distribution. First, two histograms were fit to these variables using the mixtools package. Then, the intersection point between the two histograms was determined by solving for c. Where

µ2σ12− σ2(µ1σ2 + σ1q(µ1− µ2)2 + 2(σ12− σ22)logσσ12)

c = σ12− σ22

Where µ1 and σ1 are the mean and standard deviation for the left distribution and ν2 and σ2 are the mean and standard deviation for the right distribution.

Finds where two histograms intersect and Creates dummy variables based on bimodal data:

Density curves for each bimodal predictor with two normal distributions fit to each peak.

Density Curves

0 200 400 600 800 1000 1200 1400

Data

Density Curves

0 50 100 150 200 250 300 350

Data

Density Curves

0 50 100 150 200 250

Data

Summary of bimodal dummy variable generation

Column

Name µ1 µ2 σ1 σ2 c Count of 0’s

bat_so 606.31 972.61 199.88 114.06 806.38 969

p_hr 31.43 127.37 14.39 52.08 60.93 1602

bat_hr 26.55 125.06 13.10 48.72 54.93 1583

Bar graphs for each of the bimodal dummy variables.

0 represents the amount of observations for the original variable where the value was above c, while 1 represents the amount of observations below c*

0 1 0 1

bi_bat_so bi_p_hr bi_bat_hr

Saber Model

We would like to employ outside analysis in order to engineer new, potentially powerful features. Popularized in the movie “Money ball”, the SABERMETRICS model for baseball analysis includes a feature known as BsR (base runs). This statistic estimates the amount of runs a team should score.

Base Runs (BsR) is a run estimator developed by David Smyth in the early 1990s. It was inspired by Bill James’ Runs Created, and like Runs Created it is a dynamic model of team run scoring. Many different versions of Base Runs have been introduced by several sabermetricians, to accommodate different data sets and philosophies. However, all Base Runs formulas take the form:

A*B/(B + C) + D

BSR = AB/(B + A) + C

where:

A = TEAM_BATTING_1B+TEAM_BATTING_2B+TEAM_BATTING_3B+TEAM_BATTING_BB

B = 1.02(1.4TEAM_TOTAL_BASES − 0.6TEAM_BATTING_H + 0.1TEAM_BATTING_BB)

C = TEAM_BATTING_HR

Histogram of BSR Predictor

BSR

Reviewing the correlations

After performing multiple cleaning and imputation steps, we would like to visualize again the correlations between features and their target, as well as between features themselves.

0.35

0.29 0.57

0.18 0.09 0.44

0.23 0.09 0.26 0.51

−0.04 −0.31 0.18 0.72 0.38

0.19 0.12 0.45 0.97 0.46 0.66

−0.07 −0.28 0.13 0.55 0.16 0.87 0.57

−0.03 0.08 0.31 0.51 0.40 0.29 0.49 0.14

0.13 0.32 −0.13 −0.65 −0.28 −0.68 −0.59 −0.53 −0.41

0.13 0.04 −0.18 −0.46 −0.26 −0.25 −0.42 −0.09 −0.57 0.46

0.02 0.01 −0.25 −0.52 −0.23 −0.32 −0.50 −0.24 −0.41 0.45 0.54

−0.09 0.07 −0.36 −0.75 −0.37 −0.64 −0.72 −0.45 −0.56 0.71 0.53 0.48

0.20 0.16 0.21 0.29 0.72 0.09 0.32 0.04 0.22 −0.11 −0.08 −0.16 −0.19

0.18 0.69 0.30 −0.18 −0.02 −0.46 −0.12 −0.35 −0.10 0.47 0.18 0.12 0.34 0.16

0.14 0.70 −0.06 −0.58 −0.27 −0.73 −0.54 −0.60 −0.30 0.69 0.33 0.36 0.59 −0.07 0.64

0.34 0.63 0.75 0.82 0.45 0.39 0.82 0.28 0.45 −0.32 −0.33 −0.40 −0.55 0.32 0.25 −0.09

0.32 0.55 0.68 0.88 0.49 0.46 0.87 0.33 0.49 −0.39 −0.37 −0.44 −0.61 0.33 0.17 −0.17 0.99

bat_h bat_2b bat_hr bat_bb bat_so p_hr p_so f_dp bat_3b bas_sb bas_cs f_e p_bb p_h bat_1b total_bases saber

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1

These correlation values make much more sense than before. We can see that features no longer have 1.0 correlations, which in general are highly unlikely to occur naturally. The new most correlated (and least correlated) features are as follows:

• p_hr & bat_hr (0.97): This is an interested correlation, as we would not have initially expected the amount of home runs allowed to be correlated with the number of home runs achieved from a team. However, one could make the argument that a team which focuses on offense would similarly be lacking in defense.

• bat_1b & bat_so (-0.73): These features are negatively correlated, which makes intuitive sense. If a team has many players making it to base, then conversely we would expect that this team would have less strikeouts at bat.

• bat_so & p_so (0.87): These features intuitively should not have such high correlation. Similar to above, we would not expect the performance of batter strikeouts to have any relationship to the performance of pitching strikeouts on the same team.

Target correlation plot after cleaning

After applying all transformations and imputations, we can see that the feature correlation with the target variable has also improved. Features predicted to have positive correlations (as provided by the assignment guide) do tend to have positive correlations. Similarly, features with expected negative correlations behave as described. This provides us some level of validation as we take the next steps with model building.

Build Models

Create data frame with 0 rows and 3 columns and Create function to update the tracker

Examine base model, no transformations, no engineering

Our first model (Base model) will use all of the initially provided columns, after cleaning and imputation. We will use the results of this model to understand a baseline for our future model development.

##

## Coefficients: (1 not defined because of singularities)

## Estimate Std. Error t value Pr(>|t|)

## (Intercept) 6.9253e+02 7.4378e+01 9.3110 < 2.2e-16

## bat_h 1.8255e-02 3.8878e-03 4.6955 2.819e-06

## bat_2b 2.7989e-02 8.6740e-03 3.2268 0.0012697

## bat_hr 7.1466e-02 2.8495e-02 2.5080 0.0122103

## bat_bb 3.9873e-02 4.5933e-03 8.6807 < 2.2e-16

## bat_so -1.9114e-02 4.1948e-03 -4.5566 5.476e-06

## p_hr 1.5501e-02 2.4562e-02 0.6311 0.5280260

## p_bb -5.1417e-01 1.4920e-01 -3.4461 0.0005791

## p_so 1.4940e-03 3.2302e-03 0.4625 0.6437681

## f_dp -1.0332e-01 1.3232e-02 -7.8084 8.784e-15

## bat_3b 1.4218e+00 2.0473e-01 6.9449 4.930e-12

## bas_sb 1.8485e+00 2.0479e-01 9.0267 < 2.2e-16

## bas_cs 1.3926e-01 2.5226e-01 0.5520 0.5809747

## f_e -5.0564e+02 5.6774e+01 -8.9062 < 2.2e-16

##

## n = 2276, p = 14, Residual SE = 13.37938, R-Squared = 0.28

Based on the above output, we can see that this model performs relateively poorly against the training data. However, as this is our base model, we will assess the performance of all future models against this value. Moving forward, if we can lift the Adjusted rˆ2 to above 0.3, we will consider it a general improvement.

Evaluate SABER model

The next model we would like to evaluate is the SABER model. Here we will use all original features, and additionally we will include the engineered SABER metrics. Hopefully we will see a lift in performance after utilizing these industry-derived features.

##

## Coefficients: (3 not defined because of singularities)

## Estimate Std. Error t value Pr(>|t|)

## (Intercept) 6.8848e+02 7.4296e+01 9.2667 < 2.2e-16

## bat_h -2.1195e-01 5.0614e-02 -4.1875 2.928e-05

## bat_2b -1.0044e-01 2.9629e-02 -3.3899 0.0007111

## bat_hr -7.1592e-01 1.7701e-01 -4.0446 5.416e-05

## bat_bb 2.5793e-02 5.7484e-03 4.4870 7.586e-06

## bat_so -2.5209e-02 4.4272e-03 -5.6942 1.401e-08

## p_hr -2.3380e-02 2.5598e-02 -0.9134 0.3611534

## p_so 4.6712e-03 3.2620e-03 1.4320 0.1522772

## f_dp -1.1571e-01 1.3448e-02 -8.6043 < 2.2e-16

## bat_3b 1.0546e+00 2.1917e-01 4.8115 1.597e-06

## bas_sb 1.9034e+00 2.0605e-01 9.2376 < 2.2e-16

## bas_cs 2.5214e-01 2.5180e-01 1.0014 0.3167610

## f_e -5.0468e+02 5.6876e+01 -8.8733 < 2.2e-16

## p_bb -3.8029e-01 1.5221e-01 -2.4984 0.0125459

## bi_bat_so1 -1.7174e+00 1.0639e+00 -1.6143 0.1065933

## bi_p_hr1 -3.4765e+00 1.9107e+00 -1.8195 0.0689662

## bi_bat_hr1 5.8917e+00 2.0429e+00 2.8839 0.0039649

## saber 4.8640e-01 1.0613e-01 4.5829 4.837e-06

##

## n = 2276, p = 18, Residual SE = 13.28237, R-Squared = 0.29

As expected, we did see a lift in perfomance after including SABER metrics. However, the lift was hardly significant. We are still below 0.3 Adjusted Rˆ2.

SABER reduced

Here we will test out a more parsimonious version of the above SABER model. In the spirit of simplifying the model for human use and understanding, we will select only the features that have high significance from the above SABER model. Additionally, we will exlude any features which were included as part of the construction of SABER, in order to reduce inherent multicollinearity.

## Estimate Std. Error t value Pr(>|t|)

## (Intercept) 4.8049e+02 6.8429e+01 7.0217 2.884e-12

## saber 5.2232e-02 3.1766e-03 16.4425 < 2.2e-16

## bi_bat_hr1 3.0012e-02 1.0873e+00 0.0276 0.978

## f_e -3.4661e+02 5.1743e+01 -6.6987 2.643e-11

## bas_sb 2.1296e+00 1.9113e-01 11.1425 < 2.2e-16

## f_dp -1.0406e-01 1.3536e-02 -7.6882 2.208e-14

## bat_so -2.0973e-02 1.6315e-03 -12.8553 < 2.2e-16

## bat_bb 2.7673e-02 2.8093e-03 9.8504 < 2.2e-16

##

## n = 2276, p = 8, Residual SE = 13.56093, R-Squared = 0.26

While the Adjusted Rˆ2 has been slightly reduced to 0.26, we have also significantly reduced the complexity of the model. This provides value in itself, as the model can be more easily distributed to players and coaches.

Step AIC

## Estimate Std. Error t value Pr(>|t|)

## (Intercept) 6.8681e+02 7.2984e+01 9.4104 < 2.2e-16

## bat_h 1.8176e-02 3.8290e-03 4.7469 2.195e-06

## bat_2b 2.8459e-02 8.5675e-03 3.3218 0.0009086

## bat_hr 8.6889e-02 9.9994e-03 8.6895 < 2.2e-16

## bat_bb 3.7782e-02 3.9872e-03 9.4760 < 2.2e-16

## bat_so -1.7546e-02 2.2134e-03 -7.9270 3.491e-15

## p_bb -4.5775e-01 1.3582e-01 -3.3703 0.0007635

## f_dp -1.0287e-01 1.3187e-02 -7.8004 9.342e-15

## bat_3b 1.4629e+00 2.0035e-01 7.3020 3.905e-13

## bas_sb 1.8969e+00 1.9331e-01 9.8128 < 2.2e-16

## f_e -5.0186e+02 5.5816e+01 -8.9913 < 2.2e-16

##

## n = 2276, p = 11, Residual SE = 13.37488, R-Squared = 0.28

Square Root Step AIC

The following model was generated using the same AIC methodology, except that the target variable was square rooted.

## Estimate Std. Error t value Pr(>|t|)

## (Intercept) 4.2631e+01 4.2723e+00 9.9785 < 2.2e-16

## bat_h -1.2958e-02 2.9130e-03 -4.4484 9.072e-06

## bat_2b -6.5019e-03 1.7158e-03 -3.7894 0.0001550

## bat_hr -4.6426e-02 1.0289e-02 -4.5123 6.741e-06

## bat_bb 1.7533e-03 3.1617e-04 5.5454 3.275e-08

## bat_so -9.7840e-04 1.5165e-04 -6.4517 1.349e-10

## f_dp -6.4855e-03 7.8615e-04 -8.2498 2.661e-16

## bat_3b 6.3764e-02 1.2722e-02 5.0123 5.796e-07

## bas_sb 1.1348e-01 1.2044e-02 9.4223 < 2.2e-16

## bas_cs 2.7857e-02 1.4665e-02 1.8996 0.0576168

## f_e -2.8383e+01 3.2744e+00 -8.6683 < 2.2e-16

## p_bb -3.0901e-02 8.2592e-03 -3.7414 0.0001876

## bi_bat_so1 -9.6903e-02 6.2247e-02 -1.5567 0.1196707

## bi_p_hr1 -1.8528e-01 1.0787e-01 -1.7176 0.0860074

## bi_bat_hr1 3.1635e-01 1.1674e-01 2.7099 0.0067814

## saber 3.0081e-02 6.1195e-03 4.9156 9.484e-07

##

## n = 2276, p = 16, Residual SE = 0.77728, R-Squared = 0.31

Model Selection

0.303144087087744

0.28899695594346

0.279058928565008

0.278573831105552

0.258861608098488

Base Model Saber Model Saber Reduced Step AIC Step AIC Sqrt model

The model that is ultimately chosen for this analysis is Step AIC Square Root. We were able to increase over the base model by 3%. AIC is a measure of multicollinearity so the selection process parsed out variables that were highly colinear with other variables, giving us a model that has the lowest AIC values based on a select number of predictors. This is important because this model needs to be used and understood by professionals in the industry; the step AIC model ensures that only the most prominent features are included.

The QQ plot shows that the data is centered in the middle, but there is significant amount of residuals in the middle of the distribution. This is known as the “thin tail” phenomenon. Normal distributions with “thin tails” correspond to the first quantiles occurring at larger than expected values and the last quantiles occurring at less than expected values. Notice that the “thin tailed” Q-Q plot is a reflection of a “fat tailed” Q-Q plot, which is the opposite phenomenon, across the X-Y diagonal. The Residuals vs. Fitted, and scale-location plots show that the residuals are mostly centered around the zero line. However there is some skewness as a result of outliers that are marked numerically on the plot itself. The Residuals vs. Leverage plot shows that there are no residual values that exceed Cook’s distance, which is good.

Important Metrics for Step AIC Square Root Model Important metrics for the Step AIC Square Root model are:

• R-squared: 0.3027

• F-statistic: 71.54

• RSS: 1366.88 • MSE: 0.6

• RMSE: 0.774

Predictions on Evaluation Set

The predictions were generated using the evaluation set on the Square Root Step AIC model. These predictions are provided in the predictions.csv file.

Appendix

The R code that was used in the analyses shown in this report can be viewed here:

“RMD code”.

“Rpubs”.

• Thank you.

## Reviews

There are no reviews yet.