sfirke / predicting-march-madness Goto Github PK

I may be incorrect, but should the Pred = stage_1_preds part of this code be replaced with Pred = final_preds_1?

Current:
final_preds_to_send <- final_blank %>%
dplyr::select(id) %>%
mutate(Pred = stage_1_preds)

Should Be:
final_preds_to_send <- final_blank %>%
dplyr::select(id) %>%
mutate(Pred = final_preds_1)

From my comment here: https://www.kaggle.com/c/mens-machine-learning-competition-2018/discussion/51230

I hadn't thought of testing my model on a case where two teams are equally strong. Seems like a great test for any model. When I predict a dummy game where the teams are identical, I get 0.51 when it should be 0.5, even after removing the intercept. Not good. I suspect this is due to failing to permute which team is the "lower" team. Perhaps teams with lower Kaggle numbers have historically done slightly better over the years of data I trained on and that leaked into my model.

See what names already match from Pomeroy to the Kaggle data, write that out to a .csv, edit it in Excel to complete the matches.

Looks like readr::parse_number has changed a bit. It now expects a character before converting.

I would make a PR, but I forked into a repo of mine instead of a stand-alone fork. Thanks for providing these resources!

Made a slight change to your parsing function to handle that. The solution is as follows:

process_ken_pom_sheet <- function(dat){
  dat <- dat %>%
    clean_names() %>%
    mutate(seed = str_extract(x2, "[0-9]+")) %>% # extract seed where applicable
    mutate(x2 = gsub(" [0-9]+", "", x2)) # remove seed from school name
  names(dat) <- c("rank", "team", "conf", "wins_losses",
                  "adj_EM", "adj_offensive_efficiency", "adj_offensive_efficiency_seed",
                  "adj_defensive_efficiency", "adj_defensive_efficiency_seed",
                  "adj_tempo", "adj_tempo_seed", "luck", "luck_seed", "sos_adj_em",
                  "sos_adj_em_seed", "opposing_offenses", "opposing_offenses_seed",
                  "opposing_defenses", "opposing_defenses_seed", "ncsos_adj_em",
                  "ncsos_adj_em_seed", "year", "seed")
  
  dat <- dat[-1, ] %>%
    select(rank, everything()) %>%
    filter(!is.na(rank), !rank %in% c("Rank", "Rk")) %>%
    mutate(rank = as.numeric(rank)) %>%
    mutate_all(as.character) %>% 
    mutate_at(vars(adj_EM:year), parse_number) %>%
    mutate(seed = as.numeric(seed))
  dat
}

It should help the home team, hurt the away team, and be 0 on neutral turf. With the three level factor I have currently, the training model assigns some value to neutral, i.e., neutral turf slightly changes the odds of the lower team winning. This is nonsense.

Maybe switch to two binary variables, homecourt_adv_lower and homecourt_adv_higher - they would be 1/0 and 0/1, except for when they're 0/0 to represent neutral?

Since I treat all tournament games as "neutral", the tracking of homecourt in the training model is just to cancel out its effect on learning about the impact of ratings, etc.

See comment from Malte here: https://www.kaggle.com/c/mens-machine-learning-competition-2018/discussion/51230

Perhaps better to use glm() from base R where it is simple than to do this in caret::train`.

Right now the interim version with team names, Vegas / 538 comparisons, etc. is not outputted. So if you want it later to do say, comparative log-loss analysis, you have to recreate it.