predict.Rmd

---
title: "NHL Stanley Cup Playoff Predictions"
author: "Ilari Scheinin"
output: html_document
---

Previous: [3. Train models](model.html)  
Next: [README](https://github.com/ilarischeinin/stanley)

4. Make predictions
-------------------

In the previous step, five different types of models (glm, lda, nnet, rf,
svmLinear) were fitted to the training data of NHL seasons from 2003 to 2014.
Now I want to use those models to make predictions for this year.

Start by loading packages:

```{r, echo=FALSE}
options(width=120)
```

```{r}
suppressMessages({
  library(plyr)
  library(dplyr)
  library(caret)
})
```

Load the models and summary statistics.

```{r}
gamestats <- readRDS("processed.rds")
models <- readRDS("models.rds")
```

Any one of the five models could be used to make predictions. One could for
example pick the one with the highest accuracy or
[Kappa statistic](https://en.wikipedia.org/wiki/Cohen%27s_kappa). Each model contains these metrics for each combination of parameter values
that were evalauted through cross-validation. The parameters with the highest
accuracy were chosen for fitting the final model. The following function
returns the cross-validated metric for the final model.

```{r}
metric_for_final_parameters <- function(fit, metric="Accuracy") {
  if (fit$maximize) {
    chosen <- which.max(fit$result[, fit$metric])
  } else {
    chosen <- which.min(fit$result[, fit$metric])
  }
  fit$result[chosen, metric]
}

which.max(sapply(models, metric_for_final_parameters))
which.max(sapply(models, metric_for_final_parameters, metric="Kappa"))
```

We can also look at the performance over all resamplings.

```{r}
resamps <- resamples(models)
summary(resamps)

bwplot(resamps, layout=c(2, 1))
dotplot(resamps, metric="Accuracy")
dotplot(resamps, metric="Kappa")
```

So, if I were to choose one of the models, overall the random forest could be
a god choice. But instead of picking just one, here I will use all five, and
then take a majority vote on their individual predictions. Here is the function
to do that:

```{r}
winners <- function(games, models, gamestats) {
  suppressMessages({ captured <- capture.output({
    predictions <- as.data.frame(sapply(models, predict,
      newdata=add_stats(games, gamestats)))
  })})
  if (nrow(games) == 1) {
    predictions <- as.data.frame(t(predictions))
    row.names(predictions) <- NULL
  }
  predictions$winner <- apply(predictions, 1, function(x)
    names(sort(table(x), decreasing=TRUE))[1])
  predictions <- ifelse(predictions == "away", games$awayteam, games$hometeam)
  cbind(games, predictions)
}
```

I also need to include the same `add_stats()` function that was used when
training the models.

```{r}
add_stats <- function(games, gamestats, which=c("both", "single", "overall")) {
  which <- match.arg(which)
  if (which == "overall") {
    away <- left_join(games, gamestats[["overall"]],
      by=c("season", awayteam="team"))
    home <- left_join(games, gamestats[["overall"]],
      by=c("season", hometeam="team"))
  } else {
    away <- left_join(games, gamestats[["away"]],
      by=c("season", awayteam="team"))
    home <- left_join(games, gamestats[["home"]],
      by=c("season", hometeam="team"))
  }

  if (which == "both") {
    away2 <- left_join(games, gamestats[["home"]],
      by=c("season", awayteam="team"))
    home2 <- left_join(games, gamestats[["away"]],
      by=c("season", hometeam="team"))
  }

  games$goals <- away$goals - home$goals
  games$shots <- away$shots - home$shots
  games$faceoffs <- away$faceoffs - home$faceoffs
  games$penalties <- away$penalties - home$penalties
  games$pp <- away$pp - home$pk
  games$pk <- away$pk - home$pp

  if (which == "both") {
    games$goals2 <- away2$goals - home2$goals
    games$shots2 <- away2$shots - home2$shots
    games$faceoffs2 <- away2$faceoffs - home2$faceoffs
    games$penalties2 <- away2$penalties - home2$penalties
    games$pp2 <- away2$pp - home2$pk
    games$pk2 <- away2$pk - home2$pp
  }

  games
}
```

Before I can go on to the predictions, I need to define who is playing against
who. Here I define the actual playoff games for round 1, and all possible
combinations of the playoff bracket for subsequent rounds.

```{r}
round1_games <- data_frame(season="20142015",
  awayteam=c("PIT", "OTT", "DET", "NYI", "WPG", "MIN", "CHI", "CGY"),
  hometeam=c("NYR", "MTL", "T.B", "WSH", "ANA", "STL", "NSH", "VAN"))

round2_possibilities <- data_frame(season="20142015",
  awayteam=c(
		"WSH", "NYI", "T.B", "DET", "OTT", "OTT", "PIT", "PIT",
    "VAN", "CGY", "NSH", "CHI", "MIN", "MIN", "WPG", "CGY"),
  hometeam=c(
		"NYR", "NYR", "MTL", "MTL", "T.B", "DET", "WSH", "NYI",
    "ANA", "ANA", "STL", "STL", "NSH", "CHI", "VAN", "WPG"))

round3_possibilities <- data_frame(season="20142015",
  awayteam=c(
    "MTL", "T.B", "DET", "OTT",
    "WSH", "NYI", "PIT",
    "WSH", "NYI", "PIT",
    "DET", "OTT",
    "DET", "OTT",
    "PIT",
    "PIT",
    "STL", "NSH", "CHI", "MIN",
    "VAN", "WPG", "CGY",
    "VAN", "WPG", "CGY",
    "VAN", "WPG", "CGY",
    "MIN",
    "WPG", "CGY"),
  hometeam=c(
    "NYR", "NYR", "NYR", "NYR",
    "MTL", "MTL", "MTL",
    "T.B", "T.B", "T.B",
    "WSH", "WSH",
    "NYI", "NYI",
    "DET",
    "OTT",
    "ANA", "ANA", "ANA", "ANA",
    "STL", "STL", "STL",
    "NSH", "NSH", "NSH",
    "CHI", "CHI", "CHI",
    "VAN",
    "MIN", "MIN"))

round4_possibilities <- data_frame(season="20142015",
  awayteam=c(
    "ANA", "STL", "NSH", "CHI", "VAN", "MIN", "WPG", "CGY",
    "ANA", "STL", "NSH", "CHI", "VAN", "MIN", "WPG", "CGY",
    "T.B", "WSH", "NYI", "DET", "OTT", "PIT",
    "T.B", "WSH", "NYI", "DET", "OTT", "PIT",
    "NSH", "CHI", "VAN", "MIN", "WPG", "CGY",
    "WSH", "NYI", "DET", "OTT", "PIT",
    "WSH", "NYI", "DET", "OTT", "PIT",
    "WSH", "NYI", "DET", "OTT", "PIT",
    "MIN", "WPG", "CGY",
    "MIN", "WPG", "CGY",
    "DET", "OTT", "PIT",
    "WPG", "CGY",
    "WPG", "CGY",
    "PIT",
    "CGY"),
  hometeam=c(
    "NYR", "NYR", "NYR", "NYR", "NYR", "NYR", "NYR", "NYR",
    "MTL", "MTL", "MTL", "MTL", "MTL", "MTL", "MTL", "MTL",
    "ANA", "ANA", "ANA", "ANA", "ANA", "ANA",
    "STL", "STL", "STL", "STL", "STL", "STL",
    "T.B", "T.B", "T.B", "T.B", "T.B", "T.B",
    "NSH", "NSH", "NSH", "NSH", "NSH",
    "CHI", "CHI", "CHI", "CHI", "CHI",
    "VAN", "VAN", "VAN", "VAN", "VAN",
    "WSH", "WSH", "WSH",
    "NYI", "NYI", "NYI",
    "MIN", "MIN", "MIN",
    "DET", "DET",
    "OTT", "OTT",
    "WPG",
    "PIT"))
```

And then finally to the predictions:

```{r}
round1 <- winners(round1_games, models, gamestats)
round1
```

Now that I have predictions for round 1 winners, I pick only their games from
all the possibilities for round 2:

```{r}
round2 <- winners(
  filter(round2_possibilities,
    awayteam %in% round1$winner, hometeam %in% round1$winner),
  models, gamestats)
round2
```

Similarly for round 3, the conference finals:

```{r}
round3 <- winners(
  filter(round3_possibilities,
    awayteam %in% round2$winner, hometeam %in% round2$winner),
  models, gamestats)
round3
```

And finally the Stanley Cup final:

```{r}
round4 <- winners(
  filter(round4_possibilities,
    awayteam %in% round3$winner, hometeam %in% round3$winner),
  models, gamestats)
round4
```

My prediction for the 2015 Stanley Cup winner is **Chicago Blackhawks**.

### Validation

As the 2014-2015 playoffs are now over, we have our natural validation set
available. (Since I could have naturally tampered with the prediction process
after the fact to try to get to the real outcome, here is a link to the first
GitHub
[commit](https://github.com/ilarischeinin/stanley/blob/30c3838f629261796895ecd271d327fb7cabd543/predict.html)
that included my predictions. It was made on April 23rd. So, not before the
playoffs started on April 15th, but when round 1 was already 3-4 games in,
depending on the series.)

The prediction of Chicago Blackhawks as the the Stanley Cup winner turned out
to be correct. But in the finals they played against Tampa Bay Lightning, not
New York Rangers. So, let us look at predictions for the playoff series that
actually ended up happening. Round 1 is of course as above, but my model got two
of those series wrong. Instead of New York Islanders and St. Louis Blues, it was
Washington Capitals and Minnesota Wild who made it to the second round.

```{r}
round2_games <- data_frame(season="20142015",
  awayteam=c("WSH", "T.B", "CGY", "MIN"),
  hometeam=c("NYR", "MTL", "ANA", "CHI"))
winners(round2_games, models, gamestats)
```

Here one of the predictions was wrong. Instead of Montreal Canadians, it was
Tampa Bay Lightning who made it to the Eastern Conference final.

```{r}
round3_games <- data_frame(season="20142015",
  awayteam=c("T.B", "CHI"),
  hometeam=c("NYR", "ANA"))
winners(round3_games, models, gamestats)
```

Again, one of the predictions was wrong. In the Stanley Cup final, Chicago
Blackhawks faced Tampa Bay Lightning, not New York Rangers.

```{r}
round4_games <- data_frame(season="20142015",
  awayteam=c("CHI"),
  hometeam=c("T.B"))
winners(round4_games, models, gamestats)
```

So overall, the prediction accuracy is:  
round 1: 6 out of 8  
round 2: 3 out of 4  
round 3: 1 out of 2  
round 4: 1 out of 1  
total: 11 / 15 = 73.3 %

So, by taking the majority vote from five different types of statistical models
(glm, lda, nnet, rf, svmLinear) I was able to get a correct prediction for the
2015 Stanley Cup winner, and a 73% overall accuracy for the individual palyoff
series. An obvious next step could be to look at each one of the five models
separately to see how they performed individually.

Next: [README](https://github.com/ilarischeinin/stanley)  
Previous: [3. Train models](model.html)