Opened 3 years ago

Last modified 3 years ago

## #2893 new feature request

# Bayesian Nonlinear Regression

Reported by: | gkronber | Owned by: | |
---|---|---|---|

Priority: | low | Milestone: | HeuristicLab 4.x Backlog |

Component: | Algorithms.DataAnalysis | Version: | branch |

Keywords: | Cc: |

### Description

Currently, HL supports non-linear regression which uses the LM algorithm to fit parameters to a given non-linear model. The trained model can be used to calculate a predicted value for new inputs.

This is closely related to constants optimization in symbolic regression models, as we are using the same implementation for both.

It would be interesting to provide a feature for 'Bayesian non-linear regression'. The algorithm should determine a posterior distribution for the parameters given the data. The trained model can then be used to calculate a predictive distribution for new inputs.

It is often not possible to determine the posterior distribution analytically (depends on the structure of the model). An possible solution might be to use MCMC sampling to to estimate the posterior distribution. The samples from the posterior can than be used to calculate an estimated predictive distribution for each input.

Hamiltonian Monte Carlo seems to work well for similar tasks (see e.g. https://arxiv.org/pdf/1206.1901.pdf and http://mc-stan.org/)

### Change History (4)

### comment:1 Changed 3 years ago by gkronber

### comment:2 Changed 3 years ago by gkronber

r15748: implemented a first version of Bayesian non-linear regression using HMC sampling

### comment:3 Changed 3 years ago by gkronber

r15749: normalized line endings

### comment:4 Changed 3 years ago by gkronber

r15750: added scaling by number of rows to make leapfrog integration independent from the number of rows

**Note:**See TracTickets for help on using tickets.

r15747: created a branch