Gradient boosting meta-learner for regression and classification

[gkronber]

It would be nice to support a kind of boosting where multiple models are learned step by step and the weight of observations is adapted based on the residuals of the models learned so far.

Friedmans "Stochasic Gradient Boosting" (1999) could be implemented for regression and classification problems.

Since version 3.3.12 there is a specific implementation of gradient boosted trees. It would be great if we could also implement gradient boosting as a meta learner that uses any regression algorithm as the base learner.