Free cookie consent management tool by TermsFeed Policy Generator

Opened 6 years ago

Last modified 3 years ago

#2893 assigned feature request

Bayesian Nonlinear Regression

Reported by: gkronber Owned by: gkronber
Priority: low Milestone: HeuristicLab 4.x Backlog
Component: Algorithms.DataAnalysis Version: branch
Keywords: Cc:

Description

Currently, HL supports non-linear regression which uses the LM algorithm to fit parameters to a given non-linear model. The trained model can be used to calculate a predicted value for new inputs.

This is closely related to constants optimization in symbolic regression models, as we are using the same implementation for both.

It would be interesting to provide a feature for 'Bayesian non-linear regression'. The algorithm should determine a posterior distribution for the parameters given the data. The trained model can then be used to calculate a predictive distribution for new inputs.

It is often not possible to determine the posterior distribution analytically (depends on the structure of the model). An possible solution might be to use MCMC sampling to to estimate the posterior distribution. The samples from the posterior can than be used to calculate an estimated predictive distribution for each input.

Hamiltonian Monte Carlo seems to work well for similar tasks (see e.g. https://arxiv.org/pdf/1206.1901.pdf and http://mc-stan.org/)

Change History (5)

comment:1 Changed 6 years ago by gkronber

r15747: created a branch

comment:2 Changed 6 years ago by gkronber

r15748: implemented a first version of Bayesian non-linear regression using HMC sampling

comment:3 Changed 6 years ago by gkronber

r15749: normalized line endings

comment:4 Changed 6 years ago by gkronber

r15750: added scaling by number of rows to make leapfrog integration independent from the number of rows

comment:5 Changed 3 years ago by gkronber

  • Owner set to gkronber
  • Status changed from new to assigned
Note: See TracTickets for help on using tickets.