Opened 19 months ago

Last modified 19 months ago

#2893 new feature request

Bayesian Nonlinear Regression

Reported by: gkronber Owned by:
Priority: low Milestone: HeuristicLab 4.x Backlog
Component: Algorithms.DataAnalysis Version: branch
Keywords: Cc:

Description

Currently, HL supports non-linear regression which uses the LM algorithm to fit parameters to a given non-linear model. The trained model can be used to calculate a predicted value for new inputs.

This is closely related to constants optimization in symbolic regression models, as we are using the same implementation for both.

It would be interesting to provide a feature for 'Bayesian non-linear regression'. The algorithm should determine a posterior distribution for the parameters given the data. The trained model can then be used to calculate a predictive distribution for new inputs.

It is often not possible to determine the posterior distribution analytically (depends on the structure of the model). An possible solution might be to use MCMC sampling to to estimate the posterior distribution. The samples from the posterior can than be used to calculate an estimated predictive distribution for each input.

Hamiltonian Monte Carlo seems to work well for similar tasks (see e.g. https://arxiv.org/pdf/1206.1901.pdf and http://mc-stan.org/)

Change History (4)

comment:1 Changed 19 months ago by gkronber

r15747: created a branch

comment:2 Changed 19 months ago by gkronber

r15748: implemented a first version of Bayesian non-linear regression using HMC sampling

comment:3 Changed 19 months ago by gkronber

r15749: normalized line endings

comment:4 Changed 19 months ago by gkronber

r15750: added scaling by number of rows to make leapfrog integration independent from the number of rows

Note: See TracTickets for help on using tickets.