Evolution of mean and covariance functions for Gaussian processes using GP

[gkronber]

r8751:8755 initial import of GPR evolution branch

[gkronber]

r8826: added benchmark instances for GPR tuning

[gkronber]

r8873, r8879 implemented possible benchmark instances for covariance evolution

[gkronber]

r9099: adapted GP tuning branch to work with most recent trunk

[gkronber]

r9100: added build script

[gkronber]

r9102: ILNumerics source for experimentation

[gkronber]

r9103: set svn:ignore properties

[gkronber]

r9124: implemented utility app to draw random samples using a GP prior

[gkronber]

r9125: svn:ignore

[gkronber]

r9212: worked on Gaussian Process evolution problem

[gkronber]

r9338: minor adaptations necessary for the EuroCAST presentation.

[gkronber]

r9387: added CovNN symbol and tree node

[gkronber]

r9562: implemented GP model using the Eigen library for calculation of model likelihood and gradients r9566: removed auto-generated files and set svn:ignore properties

[gkronber]

r9622: fixed generation of GPR problem instances (sampling from Gaussian processes) to work together with the current trunk version

[gkronber]

r10756:10757

• adaptations to work with current trunk version
• removed obsolete classes and cleaned up the implementation.
• Added the ProblemInstanceProvider interface to allow loading of CSV files.

[gkronber]

r12946: added implementation of synthesis of covariance functions for Gaussian Process regression as a BasicProblem<SymbolicExpressionTreeEncoding>

[gkronber]

r12948: added missing plugin reference

[gkronber]

r12953: deleted obsolete branch (functionality has been partially implemented in trunk version of HeuristicLab) ILNumerics, Eigen, and MKL implementations are also removed.

[gkronber]

Only r12946, r12948, r12953 need to be reviewed. The implementation in trunk uses nothing from the branch.

[gkronber]

r12954: removed unused usings

[mkommend]

Review comments of GaussianProcessCovarianceOptimizationProblem:

• Line 254: Is there no more specific expection available as Exception?
• Why is the resulting GP model not displayed?
• The resulting covariance function should be built so that it can be reused for GP modeling.

[gkronber]

Replying to mkommend:

Review comments of GaussianProcessCovarianceOptimizationProblem:

• Line 254: Is there no more specific expection available as Exception?

Ok, can do.

• Why is the resulting GP model not displayed?
• The resulting covariance function should be built so that it can be reused for GP modeling.

I think the only way to accomplish this would be to introduce problem state (maybe as parameters) in order to transmitt information from Evaluate() to Analyze(). Other suggestions?

[gkronber]

r13200: also added the Gaussian process solution as a result

[gkronber]

r13201: create a copy of the hyper-parameter array

[mkommend]

r13209: Implemented IStatefulItem in GaussianProcessCovarianceOptimizationProblem to clear the temporary parameters correctly.

[mkommend]

Reviewed r13200 and r13201 and corrected the behavior with r13209.

I noticed is that although the seed is fixed the algorithm gives different results. I don't know if this behavior was introduced with the changesets mentioned above or existed before.

[gkronber]

Thanks for spotting this bug! This is a generally a critical problem when using BasicProblems in combination with ParallelEngine. (see #2511)

[gkronber]

r13234: synchronized access to PRNG in GaussianProcessCovarianceOptimizationProblem.Evaluate() and added a comment

[mkommend]

r13242: Introduced separate lock object for synchronizing access to the random object.

Locking on this or passed references is considered a bad practise.

[gkronber]

By mistake the changeset r13160 (see #2504) slipped through (which includes changes on files all over the solution)!

[gkronber]

r13286: merged r12946,r12948,r12953,r12954 from trunk to stable

[gkronber]

r13287: partial merge of r13160 from trunk to stable

[gkronber]

r13290: merged r13200,r13201,r13209,r13234,r13242 from trunk to stable