Model comparisons
- CV MSE/RMSE, Log loss/Accuracy/Brier Score
- Compare with previous best model, or a simpler but reasonable model. If using some regression or classification model that is fancier than linear or logistic regression, compare with linear or logistic regression.
- If the models have coefficients, do they have the expected sign?
- See the variable importance section. Can compare the importance of variables across models.
- If the models have predictions, compare the predictions. What observations have the biggest differences?
- If the models have uncertainties, compare the uncertainties.