Boosting support for classification and regression algorithms

[gkronber]

It would be nice to support a kind of boosting where multiple models are learned step by step and the weight of observations is adapted based on the residuals of the models learned so far. See ​AdaBoost.

AdaBoost with sampling instead of weighting could be implemented for classification problems.

Friedmans "Stochasic Gradient Boosting" (1999) could be implemented for regression and classification problems.

One important question is where the gradient for our target functions is implemented. In gradient boosting we need to use the correct gradient which is appropriate for the currently selected fitness function.