Prediction

Feature selection

• Feature Selection Approaches from http://r-statistics.co
• Leave-one-out feature importance (LOFO) + LightGBM

Boruta

• Boruta Explained Exactly How You Wished Someone Explained to You
• Random Forest Feature Selection

Random forest

• https://en.wikipedia.org/wiki/Random_forest
• randomForest package
  • Error: protect(): protection stack overflow. The trick works on my data with 26 obs and 28110 variables.
  • A complete guide to Random Forest in R
  • 8.5 Permutation Feature Importance from Interpretable Machine Learning by Christoph Molnar
  • 15 Variable Importance from caret
  • How to find the most important variables in R
  • ?importance
  • Random Forest Variable Importance and a simple example with 2 correlated predictors.
  • If you just print the importance object from the model they are the raw importance values. However, when you use the importance function, the default for the scale argument is TRUE which returns the importance values divided by the standard error.
  • Measures of variable importance in random forests
  • Difference between varImp (caret) and importance (randomForest) for Random Forest
  • Utilizing Machine Learning algorithms (GLMnet and Random Forest models) for Genomic Prediction of a Quantitative trait
• Gene selection and classification of microarray data using random forest 2006, and the R package varSelRF
  • The most reliable measure is based on the decrease of classification accuracy when values of a variable in a node of a tree are permuted randomly
  • this measure of variable importance is not the same as a non-parametric statistic of difference between groups, such as could be obtained with a Kruskal-Wallis test)
• Feature Importance in Random Forest

Gradient boost

• https://en.wikipedia.org/wiki/Gradient_boosting
• Gradient boosting in R

GBDT: Gradient Boosting Decision Trees

• An Introduction to Gradient Boosting Decision Trees
• GBDTs & Random Forests As Feature Transformers