Implement Naive Grammar Enumeration for Symb. Regression

[lkammere]

As a first step in deterministic symbolic regression, implement an algorithm that iterates and checks all possible sentences of a grammar in order to find the best model structure for (very simple) regression problems.

[lkammere]

r15712: Add basic class structure, grammar and grammar iteration.

r15714: Add tree hashing for addition and multiplication.

r15722: Add evaluation of sentences.

r15723: Add simple data analysis tests and further informations about algorithm run.

r15724: Add parsing to infix form for debugging purpose.

r15725: Refactor tree hash function.

r15726: Overwrite long sentences when a shorter one with same hash was found.

r15734: worked on grammar enumeration

r15746: Refactor grammar enumeration alg.

r15765: Add graphviz output.

[lkammere]

r15772: Extend grammar enumeration algorithm's grammar to exp, log and sine.

r15773: Update unit tests to cover problems with exp, log and sine.

[lkammere]

r15776: Refactor data generation in unit tests.

[lkammere]

r15784: Add basic implementation for inverse factors.

[gkronber]

r15806: made a few comments

[lkammere]

r15791: Replace integer hashing of phrases with simplification to (temporary) string representations.

r15795: Remove nested divisions from grammar and hashing.

r15800: Refactor code and fix performance issues.

r15803: Deactivate generation of dot file for visualizing search tree.

r15812: Performance Improvements - Only store hash of archived phrases and reduce number of enumerators.

[lkammere]

r15817: Add cosine to grammar.

[lkammere]

r15821: Move code for visualization and logging of sentences to separate classes.

r15823: Fixed build settings.

r15824: Move R² calculation of sentences to separate class and allow its deactivation.

[lkammere]

r15825: Added prebuild event.

r15827: Change implementation of symbol strings from list to array.

r15828: Implement IEquatable interface in symbols. Minor performance improvements.

[lkammere]

r15832: Fix Equals methods in Symbols, Move semantical hashing of phrases to separate class.

r15834: Store production rules in grammar instead of nonterminal symbols.

r15835: Split huge hashing function into smaller ones.

[gkronber]

r15840 added utility console program for clustering of expressions (work in progress)

[gkronber]

r15841: fixed FLANNParameters structure

[gkronber]

r15842: added clustering of functions and output of clusters, fixed bug in evaluation

[lkammere]

r15843: Remove duplicates in logged sentences using bash commands.

r15849: Add constants to grammar.

r15850: Remove cosine from grammar.

r15851: Remove cosine terminal symbols.

[lkammere]

r15860: Change complexity measure from number of nodes in tree to number of variable references.

r15861: Make constant optimization toggleable in algorithm.

[lkammere]

r15883: Priorize phrases whose (fully expanded) terms result in high R².

[lkammere]

r15903: worked on cluster analysis / visualization for GPTP

r15907: Changes in search heuristic for solving Poly-10 problem. Adapt tree evaluation to cover non-terminal symbols.

r15910: Fix length parameter when priorizing phrases and add weighting parameter to control exploration/exploitation during search, fix copy constructors in Analyzers.

[gkronber]

r15911: Changed initialization of SentenceLogger because started event is not called after Run(). Logging to GZipStream.

[gkronber]

r15924: remove obsolete code in C# program for the evaluation of sentences, switch to NSME as quality measure. Tried plotting functions within clusters in R

[bburlacu]

r15949: Fix serialization (saving the algorithm).

[bburlacu]

r15950: Try to use variable importance information (from a random forest) to guide the search.

[bburlacu]

r15957: Minor refactor; fix multiple analyzer event registration

[bburlacu]

r15960: Fix serialization and cloning and plugin properties.

[bburlacu]

r15963: Add missing storable constructors

[bburlacu]

r15965: Improve hashing performance (about 10% measured improvement)

[bburlacu]

r15974:

• implement LRU cache for storing search nodes (needs better integration with the algorithm main loop)
• introduce SortedSet for handling priorities (better memory usage, possibility to remove bad priorities, slight performance penalty)
• fix serialization and cloning

[bburlacu]

r15975: address additional serialization issues, make Production implement IList<T> (instead of deriving from List<T>)

[bburlacu]

r15977: Clear search data structures at the end of the run (huge memory savings)

[bburlacu]

r15979: Register algorithm events after deserialization.

[lkammere]

r15981: Refactor properties to comply with .NET 4.5.2

[bburlacu]

r15982: Add storable constructors for analyzers

[bburlacu]

r15985: Simplify code in RSquaredEvaluator. Turn on linear scaling for the constant optimization evaluator.

[bburlacu]

r15987: Make sure to clear search data structures before returning in GrammarEnumerationAlgorithm.OnStopped()

[bburlacu]

r15993: Refactor code

• move a few methods to the Grammar class
• use a plain dictionary for storing search nodes in the SearchDataStore (instead of LRU cache)
• make it easier to keep a consistent state between the algorithm and the evaluator (optimize constants flag)
• track trajectories in quality/length space for best solutions
• remove variable importances for now

[bburlacu]

r15994: Add symbolic regression solution to results during algorithm run and scale model.

[bburlacu]

r16019: Fix properties lacking implementation in Production.

[bburlacu]

r16022: Remove MaxSentenceLength from priority calculation for the time being.

[bburlacu]

r16026:

• replace functionally-overlapping classes Production and SymbolString with a single class SymbolList
• refactor methods from Grammar class as methods and properties of SymbolList
• add parameter for the number of constant optimization iterations
• refactor code

[bburlacu]

r16053: Refactor RSquaredEvaluator as a standalone ParameterizedNamedItem which is a parameter of the algorithm. Implement BestSolutionAnalyzer analyzer for quality statistics. Add license headers where missing.

[bburlacu]

r16056: Remove redundant EvaluatePhrase method in the Grammar class and fix compilation of tests.

[bburlacu]

r16073: Implement restarts for constant optimization in the RSquaredEvaluator

[lkammere]

r16088: Store pareto-optimal sentences (quality/complexity) to grammar enumeration.

[lkammere]

r16090: Explicitely store all pareto-optimal RegressionSolution objects at the end of the algorithm.

[gkronber]

r16151: deleted obsolete files

[bburlacu]

r16157:

• Update IGrammarEnumerationEvaluator interface (add Evaluate method accepting an ISymbolicExpressionTree for the case when the constants have already been optimized in the tree, add boolean OptimizeConstants flag),

• small refactor in GrammarEnumeration/GrammarEnumerationAlgorithm.cs

• add unit tests

[bburlacu]

r16159: Refactor unit test using only C# 4.5 features.

[bburlacu]

r16176: Fix hashing

[bburlacu]

r16193: Implement new hasher (faster & less collision prone) and update unit tests.

r16194: Fix compilation errors in test :(

[lkammere]

r16198: Cleanup R-scripts for expression clustering.

r16201: remove branch as it was migrated to Git.