Other types of regression models
- Nice neg bin example using MLE? https://stats.stackexchange.com/questions/384932/regarding-glm-nb-and-my-parameter
- Multinomial regression. Like logistic regression, but for categorical outcomes with 2 or more categories. See ROS, 15.5. R package:
polr. For multi-class log loss you can use the
MultiLogLoss from the MLmetrics package. See here
- Zero-inflated Poisson or Negative Binomial regression. Can be useful for count outcomes where there are more zeros than predicted by a (standard) Poisson or negative binomial regression model. This could potentially be used for the EV charging station data (we removed all census tracts with zero charging stations). See ROS 15.8 and BMLR 4.10. R Package:
zeroinfl.
- Harmonic regression. Fit the best sine/cosine curve to data. https://rdrr.io/cran/HarmonicRegression/man/harmonic.regression.html
- Beta regression. When outcome has values in the open interval (0,1) (not just 0 and 1, the open interval (0,1) and can be assumed to be y ~ Beta(, ). Like when the outcome is a proportion. Generalizable to an interval (a,b) using transformations. https://cran.r-project.org/web/packages/betareg/vignettes/betareg.pdf.
- Dirichlet regression. Kind of like multinomial logit regression is a generalization of logistic regression, this is a generalization of Beta regression. Model the proportions of two or more outcomes, proportions sum to 1. Package: https://cran.r-project.org/web/packages/DirichletReg/DirichletReg.pdf
- Spline regression. See ISLR https://www.statlearning.com/. Piecewise polynomial.
- Generalized Additive Models. See ISLR https://www.statlearning.com/. The workshop notes https://r.qcbs.ca/workshop08/book-en/ have some examples.
- Discussion of Ridge, Lasso, and Bayes.
https://ekamperi.github.io/mathematics/2020/08/02/bayesian-connection-to-lasso-and-ridge-regression.html
- Bayesian regression. https://stat.columbia.edu/~gelman/book/
- Splines. Piecewise polynomials with continuity constraints. See ISLR https://www.statlearning.com/ Section 7.4 Regression Splines, Section 7.5 Smoothing Splines, and 7.8.2 (lab exercises).
- Ordinal regression. Function
polr in MASS.
- Regularized ordinal regression. https://stackoverflow.com/questions/75790997/why-do-the-coefficents-from-orginal-regularized-regression-ordinalnet-have-the
- Mixed effects ordinal regression. https://stats.stackexchange.com/questions/238581/how-to-use-ordinal-logistic-regression-with-random-effects
- Random Forest. Packages
randomForest, and maybe Rborist, ranger, h20ai. Some of these behaved weirdly, if I recall correctly.
- Non-linear.
nls and nlme (non-linear mixed effects), see example here: https://www.datacamp.com/tutorial/r-formula-tutorial
- Mixed Effects Random Forest - https://rdrr.io/github/nguforche/MEml/man/MErf.html or https://rdrr.io/cran/MixRF/man/MixRF.html