Opened 3 years ago
Last modified 5 months ago
#2893 assigned feature request
Bayesian Nonlinear Regression
Reported by: | gkronber | Owned by: | gkronber |
---|---|---|---|
Priority: | low | Milestone: | HeuristicLab 4.x Backlog |
Component: | Algorithms.DataAnalysis | Version: | branch |
Keywords: | Cc: |
Description
Currently, HL supports non-linear regression which uses the LM algorithm to fit parameters to a given non-linear model. The trained model can be used to calculate a predicted value for new inputs.
This is closely related to constants optimization in symbolic regression models, as we are using the same implementation for both.
It would be interesting to provide a feature for 'Bayesian non-linear regression'. The algorithm should determine a posterior distribution for the parameters given the data. The trained model can then be used to calculate a predictive distribution for new inputs.
It is often not possible to determine the posterior distribution analytically (depends on the structure of the model). An possible solution might be to use MCMC sampling to to estimate the posterior distribution. The samples from the posterior can than be used to calculate an estimated predictive distribution for each input.
Hamiltonian Monte Carlo seems to work well for similar tasks (see e.g. https://arxiv.org/pdf/1206.1901.pdf and http://mc-stan.org/)
Change History (5)
comment:1 Changed 3 years ago by gkronber
comment:2 Changed 3 years ago by gkronber
r15748: implemented a first version of Bayesian non-linear regression using HMC sampling
comment:3 Changed 3 years ago by gkronber
r15749: normalized line endings
comment:4 Changed 3 years ago by gkronber
r15750: added scaling by number of rows to make leapfrog integration independent from the number of rows
comment:5 Changed 5 months ago by gkronber
- Owner set to gkronber
- Status changed from new to assigned
r15747: created a branch