Opened 6 years ago

Last modified 2 weeks ago

#1772 accepted feature request

Evolutionary tracking of solutions within GP

Reported by: bburlacu Owned by: bburlacu
Priority: highest Milestone: HeuristicLab 3.3.x Backlog
Component: Algorithms.GeneticAlgorithm Version: 3.3.6
Keywords: Cc:

Description

In other to perform theoretical investigations on building blocks and bloat, some new functionality is required:

  • complete genealogical trees of solution candidates
  • analyze evolutionary operators and search trajectories

Change History (238)

comment:1 Changed 6 years ago by bburlacu

r7479: Implemented an initial set of features: individual ancestry, genealogy tracking via customized genetic operators and data structures.

comment:2 Changed 6 years ago by bburlacu

r7513: Cleaned up unneeded folders
r7514: Merged latest trunk changes to SymbolicExpressionTreeEncoding.
r7515: Added tracking for mutation operators.

comment:3 Changed 6 years ago by bburlacu

r7522: Merged changes from trunk, rigged the TournamentSelector to track clones, improved tracking code and genealogy graph layout.

comment:4 Changed 5 years ago by bburlacu

r7779: Implemented new View, improved functionality (tracking of fragments and operators)

comment:5 Changed 5 years ago by bburlacu

r7785: Moved tree matching functionality in separate class, implemented new tree fragments analyzer. Fixed bug in GetCutIndex method.

comment:6 Changed 5 years ago by bburlacu

r7788: Fixed bug in fragment tracing.

comment:7 Changed 5 years ago by bburlacu

r7792:

  • Removed GetCutIndex method, and corresponding index field in the GenealogyGraphNode.
  • Implemented tracking for mutated fragments.
  • Improved FindMatch method.
  • Added IterateNodesBreadth functionality to symbolic expression trees and nodes.
  • Added check conditions for clearing global tracking structures so that the 2 analyzers are not mutually exclusive anymore.
Last edited 5 years ago by bburlacu (previous) (diff)

comment:8 Changed 5 years ago by bburlacu

r7800: Fixed display of genetic fragments (received via crossover) in the GenealogyGraphView. Added parent, offspring and fragment lengths histogram.

comment:9 Changed 5 years ago by bburlacu

r7817: Fixed a small bug in the TracingSymbolicExpressionTreeCrossover. Fixed bug in fragment matching code.

comment:10 Changed 5 years ago by bburlacu

r7997: New analyzer: SymbolicExpressionTreeRelativeLengthAnalyzer. Rewrote the SymbolicExpressionTreeFragmentsAnalyzer, added generic wrapper to wrap HL objects as items.

comment:11 Changed 5 years ago by bburlacu

r8213: Performance improvements for the GenealogyGraph. Minor refactoring to VisualGenealogyGraphArc and VisualGenealogyGraphNode classes. Added new functionality to the SymbolicExpressionTreeFragmentsAnalyzer, minor refactoring in the other two analyzers. Refactored View code. Updated project references and plugin dependencies and added HeuristicLab.Problems.DataAnalysis.Symbolic to the branch.

comment:12 Changed 5 years ago by bburlacu

r8236: Refactored code in the GenealogyGraphChart and the SymbolicExpressionTreeGenealogyAnalyzer. Used gradient from HeuristicLab.Common to color the graph nodes.

comment:13 Changed 5 years ago by bburlacu

r8248: Separated instance-specific attributes of graph node objects from the generic genealogy graph, into metadata objects kept by the specific graph class which corresponds to the specific problem instance (so for instance the SymbolicExpressionTreeGenealogyGraph will keep info about node ranks and qualities etc because that info is specific to symbolic data analysis problems).

comment:14 Changed 5 years ago by bburlacu

r8249: Fixed small bug (correct counting of node ranks) in the SymbolicExpressionTreeGenealogyAnalyzer. Fixed node tool tips in the GenealogyGraphChart.

comment:15 Changed 5 years ago by bburlacu

r8555: Partial commit -- Introduced more specific graph class for symbolic expression problems.

Last edited 5 years ago by bburlacu (previous) (diff)

comment:16 Changed 5 years ago by bburlacu

r8556: Partial commit -- Modified genealogy graph chart and view to reflect changes in the class hierarchy.

Last edited 5 years ago by bburlacu (previous) (diff)

comment:17 Changed 5 years ago by bburlacu

r8557: Introduced base class and interface for tracing operators. Introduced SymbolicExpressionTreeNodeComparer interface to be implemented by node matching operators. Fixed bug in the TracingSymbolicExpressionTreeManipulator where nodes modified by the Full- and OnePoint tree shakers were not correctly detected. The SymbolicExpressionTreeNodeSimilarityComparer is now injected in the tracing genetic operators as a parameter.

comment:18 Changed 5 years ago by bburlacu

r9082: Renamed and refactored genealogy graph components. Added SymbolGraph and FPGraph (frequent pattern graph). Added evolvability and genetic operator average improvement analyzer.

comment:19 Changed 5 years ago by bburlacu

r9083: Merged HeuristicLab.Encodings.SymbolicExpressionTreeEncoding from trunk. Added Fragment class implementing IFragment interface.

comment:20 Changed 5 years ago by bburlacu

r9084: HeuristicLab.EvolutionaryTracking.Views: refactored the GenealogyGraphChart and the GenealogyGraphView. Fixed lineage display bug.

comment:21 Changed 5 years ago by bburlacu

r9237: Merged HeuristicLab.Encodings.SymbolicExpressionTreeEncoding trunk changes into the branch. Changed fragments using the tree manipulator are now tracked using the SymbolicExpressionTreeNodeComparer.

comment:22 Changed 5 years ago by bburlacu

r9238: Added base class for the fragment analyzers. Improved analyzers, added SymbolicExpressionTreeRelativeFragmentDepthAnalyzer. Added LineageExplorer.

comment:23 Changed 5 years ago by bburlacu

r9239: Minor updates to existing views. Added LineageExplorerView.

comment:24 Changed 5 years ago by bburlacu

r9241: Merged trunk changes for HeuristicLab.Problems.DataAnalysis.Symbolic. Added SymbolicDataAnalysisSolutionTextRenderer, a class for displaying symbolic expression trees in the console.

comment:25 Changed 5 years ago by bburlacu

r9246: Removed commented code from TracingSymbolicExpressionTreeManipulator.cs. Updated project file.

comment:26 Changed 5 years ago by bburlacu

r9247: Fixed serialization bug of DirectedGraph/Vertex.cs. Modifier behavior of the SymbolicExpressionTreeGenealogyGraphBuilder so that now instances of an elite individual are connected in the graph, which makes it easier to deal with lineages.

Last edited 5 years ago by bburlacu (previous) (diff)

comment:27 Changed 5 years ago by bburlacu

r9248: Added Analyzers/SymbolicExpressionTreeEvolvabilityAnalyzer.cs.

comment:28 Changed 5 years ago by bburlacu

r9249: Updated matching criteria to BoolValues in SymbolicDataAnalysisExpressionTreeMatching.cs.

comment:29 Changed 5 years ago by bburlacu

r9250: Updated LineageExplorerView and added functionality for highlighting common fragments in a given tree. Fixed view layout and element anchoring.

comment:30 Changed 5 years ago by bburlacu

r9293: Added new similarity measures and moved them to separate class SymbolicDataAnalysisExpressionTreeSimilarity.cs.

comment:31 Changed 5 years ago by bburlacu

r9296: Updated analyzers.

comment:32 Changed 4 years ago by bburlacu

r9416: Fixed serialization of genetic fragments, added interface ISymbolicExpressionTreeNodeSimilarityComparer

comment:33 Changed 4 years ago by bburlacu

r9419: Refactoring of directed graph components, added code for correctly serializing vertices and edges. Added specific building blocks analyzers and new population diversity analyzer which correctly integrates with the parallel engine.

comment:34 Changed 4 years ago by bburlacu

r9420: Improved the LineageExplorerView, added generation labels in the GenealogyGraphChart, added new visual component VisualGenealogyGraphTextLabel.

comment:35 Changed 4 years ago by bburlacu

r9421: Merged trunk changes for HeuristicLab.Optimization.Views.

comment:36 Changed 4 years ago by bburlacu

r9423: Implemented GeneticItem-based similarity measure. Renamed ISymbolicExpressionTreeNodeComparer to ISymbolicExpressionTreeNodeSimilarityComparer.

comment:37 Changed 4 years ago by bburlacu

r9424: Merged HeuristicLab.Selection.

Last edited 4 years ago by bburlacu (previous) (diff)

comment:38 Changed 4 years ago by gkronber

Please accept the ticket!

comment:39 Changed 4 years ago by bburlacu

  • Status changed from new to accepted

comment:40 Changed 4 years ago by gkronber

  • Priority changed from medium to highest

comment:41 Changed 4 years ago by bburlacu

r9834: Manually merged HeuristicLab.Encodings.SymbolicExpressionTreeEncoding project from the trunk to the branch.

comment:42 Changed 4 years ago by bburlacu

r9835: Merged remaining trunk changes into the EvolutionaryTracking branch.

comment:43 Changed 4 years ago by bburlacu

r9943: Re-added tracking code (record copies of individuals in the GlobalCloneMap)

comment:44 Changed 4 years ago by bburlacu

r9963: Merged changes from the trunk and other branches. Added new ExtendedSymbolicExpressionTreeCanvas control for the visual exploration of tree genealogies. Reorganized some files and folders.

comment:45 Changed 4 years ago by bburlacu

r9996: Removed unnecessary subproject (not included in solution). Merged changes from tree layout branch.

comment:46 Changed 4 years ago by bburlacu

r10214: Added HeuristicLab.Problems.DataAnalysis.Symbolic.Regression project

comment:47 Changed 4 years ago by bburlacu

r10215: Updated references for HeuristicLab.Problems.DataAnalysis.Symbolic.Regression project

comment:48 Changed 4 years ago by bburlacu

r10264: Created new branch for the redesigned version of the tracking plugin.

comment:49 Changed 4 years ago by bburlacu

r10265: Renamed projects from HeuristicLab.EvolutionaryTracking to HeuristicLab.EvolutionTracking

comment:50 Changed 4 years ago by bburlacu

r10266: Deleted old directories.

comment:51 Changed 4 years ago by bburlacu

r10267: Updated files in the HeuristicLab.EvolutionaryTracking project.

comment:52 Changed 4 years ago by bburlacu

r10269: Added HeuristicLab.Problems.DataAnalysis.Symbolic and HeuristicLab.Problems.DataAnalysis.Symbolic.Views and integrated some modifications from the old branch.

comment:53 Changed 4 years ago by bburlacu

r10270: Replaced HeuristicLab.Problems.DataAnalysis.Symbolic.Views with trunk version.

comment:54 Changed 4 years ago by bburlacu

r10271: Fixed namespaces.

comment:55 Changed 4 years ago by bburlacu

r10278: Implemented generic genealogy analyzer (should work with any encoding provided the proper wiring is performed in the problem class), and before/after operators for creation, mutation and crossover.

comment:56 Changed 4 years ago by bburlacu

r10285: Added SymbolicDataAnalysisGenealogyView, updated generic analyzer and operators.

comment:57 Changed 4 years ago by bburlacu

r10293: Merged trunk changes to the branch projects. Fixed some small bugs in the GenealogyGraphArc, added SymbolicDataAnalysisGenealogyAnalyzer.cs as a facade to the generic analyzer (it looks nicer in the GUI).

comment:58 Changed 4 years ago by bburlacu

r10300: Cleaned up the design of the generic genealogy analyzer and related classes, created generic genealogy graph view. Added instrumentation code to TravelingSalesmapProblem.cs allowing genealogy tracking. Merged trunk changes (instrumentation for multi operators).

comment:59 Changed 4 years ago by bburlacu

r10302:

  • Added a ViewHost in the right side of the GenealogyGraphView which displays the encoding-specific content when a GenealogyGraphNode is clicked.
  • Migrated new SymbolicExpressionTreeChart (drawing the tree using the ReingoldTilfordLayoutEngine) to the new branch
  • Copied SymbolicDataAnalysisExpressionTreeMatching.cs and SymbolicDataAnalysisExpressionTreeSimilarityCalculator.cs to the new branch

comment:60 Changed 4 years ago by bburlacu

r10347: Small changes to the GenealogyGraph. Added generic Fragment class and interface. Added the SymbolicDataAnalysisPopulationDiversityAnalyzer. Added specialized tracking operators for symbolic data analysis. Merged trunk changes.

comment:61 Changed 4 years ago by bburlacu

r10456: Merged trunk changes.

comment:62 Changed 4 years ago by bburlacu

r10457: Fixed merge error when editing conflics for HeuristicLab.Problems.DataAnalysis.Symbolic-3.4.csproj.

comment:63 Changed 4 years ago by bburlacu

r10458: Added prebuild events. Removed unused references.

comment:64 Changed 4 years ago by bburlacu

r10459: Merged uncomitted pruning code from trunk. Removed Plugin.cs file. Renamed SymbolicDataAnalysisExpressionAfterCrossoverOperator.cs and SymbolicDataAnalysisExpressionBeforeCrossoverOperator.cs

comment:65 Changed 4 years ago by bburlacu

r10462: Fixed bug in configuration of the tracking operators, fixed null reference exception when trying to access fragments in SymbolicDataAnalysisGenealogyGraphView

comment:66 Changed 4 years ago by bburlacu

r10463: Fixed remaining namespaces from EvolutionaryTracing to EvolutionTracking. Removed autogenerated files from repo.

comment:67 Changed 4 years ago by bburlacu

r10464: Maintenance commit, removed some misplaced files, updated license header.

comment:68 Changed 4 years ago by bburlacu

r10501: Merged trunk changes and added missing frame files (for HeuristicLab.EvolutionTracking and HeuristicLab.EvolutionTracking.Views).

comment:69 Changed 4 years ago by bburlacu

r10502: Removed unused files.

comment:70 Changed 4 years ago by bburlacu

r10514: Added user controls for displaying lineages and tracking building blocks with the help of the genealogy graph (work in progress).

comment:71 Changed 4 years ago by bburlacu

r10517: Got the lineageExplorerChart to display symbolic expression trees with the correct orientation (vertically flipped).

comment:72 Changed 4 years ago by bburlacu

r10524: Reverse-merged erroneous commit (booked on ticked #2076), and merged latest trunk changes.

comment:73 Changed 4 years ago by bburlacu

r10649: Merged trunk changes (related to the ReingoldTilfordLayoutEngine).

comment:74 Changed 4 years ago by bburlacu

r10650: Added new SymbolicDataAnalysisGenealogyView and added support for the tracing of building blocks (finding the constituent ancestral elements of a selected subtree).

comment:75 Changed 4 years ago by bburlacu

r10654: Some progress on tracking fragments and building blocks, still incomplete.

comment:76 Changed 3 years ago by bburlacu

r10655: Made some progress towards the visualization of building block trajectories. Added the FragmentGraphView.

comment:77 Changed 3 years ago by bburlacu

r10656: Re-added old behavior: highlight received fragment when genealogy graph node is clicked.

comment:78 Changed 3 years ago by bburlacu

r10674: Found bug in GenealogyGraph creation, attempted fix by injecting vertex guid's in the individual scopes.

comment:79 Changed 3 years ago by bburlacu

r10675: Fixed genealogy graph creation.

comment:80 Changed 3 years ago by bburlacu

r10677: Improved usage of GenealogyGraph and GenealogyGraphNode classes. Made some progress on building block tracing.

comment:81 Changed 3 years ago by bburlacu

r10685: Tracking building blocks: worked on tracking logic and on the FragmentView.

comment:82 Changed 3 years ago by bburlacu

r10725: The subtree index needs to be adjusted if it is greater than the fragment index and the fragment is different in size than the subtree it replaced.

Last edited 3 years ago by bburlacu (previous) (diff)

comment:83 Changed 3 years ago by bburlacu

r10728: Improved aspect of visual components of the FragmentGraphView.

comment:84 Changed 3 years ago by bburlacu

r10729: Changed orientation of trees in the SymbolicExpressionTreeTile. Added a graphical bounding rectangle for each tile.

comment:85 Changed 3 years ago by bburlacu

r10730: Improved FragmentGraphView.

comment:86 Changed 3 years ago by bburlacu

r10732: Some improvements on the way the genealogy graph and the lineages are drawn.

comment:87 Changed 3 years ago by bburlacu

r10746: Small improvements to FragmentGraphView, moved tracking classes to separate folder.

comment:88 Changed 3 years ago by bburlacu

r10752: Refactored subtree tracing functionality and moved it from the view into a separate static class. Shortened symbol names in the FragmentGraphView so that trees are displayed more nicely.

comment:89 Changed 3 years ago by bburlacu

r10755: Simplified genealogy graph and fragment graph creation:

  • the genealogy graph now has a 1-1 mapping between content and vertices (as opposed to 1-n as it was previously, to account for elites); this required changes to the directed graph implementation
  • the fragment graph only contains bifurcation points (where the subtree contains the fragment so tracing must be done both ways (in the root parent AND in the other parent). in the other cases, tracing is restarted from the parent genealogy graph node.

comment:90 Changed 3 years ago by bburlacu

r10797: Added text labels to SymbolicExpressionTreeTiles so that the generation number is also displayed.

comment:91 Changed 3 years ago by bburlacu

r10801: Fixed invalid cast exception in BeforeManipulatorOperator.cs

comment:92 Changed 3 years ago by bburlacu

r10822: Partially fixed fragment detection and tracing in the case of mutation.

comment:93 Changed 3 years ago by bburlacu

r10827: Added default font, pen, and brush for the graphical components (the idea is to save memory by sharing default pens and brushes - not allocating new ones all the time), added support for tracing mutation operations

comment:94 Changed 3 years ago by bburlacu

r10830: Fixed persistence errors in the GenealogyAnalyzer and the instrumented operators, tweaked display code in the GenealogyGraphChart.

comment:95 Changed 3 years ago by bburlacu

r10832: Fixed omission in SymbolicDataAnalysisProblem.cs (forgot about the AfterManipulatorOperator)

comment:96 Changed 3 years ago by bburlacu

r10833: Fixed another minor display bug concerning elite individuals. Fixed bug when saving fragments from mutation. Displayed quality as well in the SymbolicExpressionTreeTile.

comment:97 Changed 3 years ago by bburlacu

r10834: Fixed another persistence error (vertex content was missing storable attribute)

comment:98 Changed 3 years ago by bburlacu

r10835: Fixed bug in display of node labels in SymbolicExpressionTreeTile

comment:99 Changed 3 years ago by bburlacu

r10837: Fixed identification of mutation fragments.

comment:100 Changed 3 years ago by bburlacu

r10838: Improved display of trees and fragments in the FragmentGraphView by displaying the whole tree and highlighting the subtree and the fragment inside it. Changed color scheme to look nicer.

comment:101 Changed 3 years ago by bburlacu

r10839: SymbolicExpressionTreeTile: skip start symbol when displaying the tree

comment:102 Changed 3 years ago by bburlacu

r10840: Small temporary fix to index out of bounds error in FragmentGraphView.cs.

comment:103 Changed 3 years ago by bburlacu

r10884: Added license headers where they were missing. Introduced an id map to the DirectedGraph to get graph vertices based on the id injected in the scopes by the genealogy analyzer.

comment:104 Changed 3 years ago by bburlacu

r10886: Simplified GenealogyGraph (and related components) API in an effort to improve speed and memory consumption (eg., by sharing the same arc when walking the graph in both directions: parent-child and child-parent).

Last edited 3 years ago by bburlacu (previous) (diff)

comment:105 Changed 3 years ago by bburlacu

r10888: Introduced separate class for FragmentNodes and adjusted tracing code. Fixed small bug creating the genealogy graph.

comment:106 Changed 3 years ago by bburlacu

r10890: Deleted unused VisualGenealogyGraphTextLabel.cs, fixed tracking code in BeforeManipulatorOperator.cs, re-added setters for vertex in- and outArcs.

comment:107 Changed 3 years ago by bburlacu

r10897: Adjusted namespace in Plugin.cs.frame for HeuristicLab.EvolutionTracking.Views. Simplified DirectedGraph and GenealogyGraph API. Added public events for the Vertex content (so that the graph itself can be notified when the content changes and can adjust it's content-to-vertex map. Adjusted instrumented operators code to reflect api changes.

comment:108 Changed 3 years ago by bburlacu

r10903: Removed Storable property from Vertex in- and outArcs as it causes a stack overflow exception during serialization. Added a list of arcs to the DirectedGraph class which is persisted. The Vertex in- and outArcs lists are then restored by the graph after the deserialization. Renamed Nodes data member to Vertices.

comment:109 Changed 3 years ago by bburlacu

r10936: Modified GenealogyAnalyzer to remove discarded offspring selection children from the genealogy graph.

comment:110 Changed 3 years ago by bburlacu

r11015: Added partially working implementation of a bottom-up distance calculator for symbolic expression trees. Changed latex formatter to also return a map of node ids (useful when wanting to display trees side by side).

comment:111 Changed 3 years ago by bburlacu

r11016: Made some progress with the BottomUpDistanceCalculator, still not entirely correct.

comment:112 Changed 3 years ago by bburlacu

r11021: Added ExportDot method to DirectedGraph. Worked on the BottomUpDistanceCalculator.

comment:113 Changed 3 years ago by bburlacu

r11023: Small cosmetic changes in BottomUpDistanceCalculator.cs

comment:114 Changed 3 years ago by bburlacu

r11032: Minor change to the GenealogyAnalyzer (remove properties for retrieving LookupParameter values).

comment:115 Changed 3 years ago by bburlacu

r11165: Added fix to improve accuracy of bottom-up tree distance.

comment:116 Changed 3 years ago by bburlacu

r11197: Separated tree distance calculations in different classes which implement a new interface called IDistanceCalculator. The isomorphic tree distance calculates the distance based on the maximum common subtree between two symbolic expression trees, and the bottom-up tree distance returns a value based on the number of matching pairs of nodes in a bottom-up mapping. Introduced the distance calculator as a parameter in the SimilarityCalculator operator so that the diversity analyzer can use either of the two distances.

comment:117 Changed 3 years ago by bburlacu

r11208: Merged trunk changes.

comment:118 Changed 3 years ago by bburlacu

r11209: Improved performance of the BottomUpDistanceCalculator

comment:119 Changed 3 years ago by bburlacu

r11227: Added storable attributes to all the tracking operators. Added an additional method in the genealogy analyzer which computes the relative reproductive success for each individual in the population as the ratio of its offspring which make it into the next generation.

comment:120 Changed 3 years ago by bburlacu

r11231: Removed obsolete files r11232: Merged latest changes (directed graph and bottom up distance calculator) from the BottomUpDistance branch.

comment:121 Changed 3 years ago by bburlacu

r11233: Worked towards integrating the new graph api with the tracking operators.

comment:122 Changed 3 years ago by bburlacu

r11253: Ported the rest of the changes to the DirectedGraph and Vertex to the GenealogyGraph and GenealogyGraphNode. Adapted tracking operators, analyzers and views.

comment:123 Changed 3 years ago by bburlacu

r11257:

  • GenealogyAnalyzer: compute relative success ratios per individual and save them in a DataTableHistory chart and as a heat map
  • GenealogyGraph: remove rank if it contains no vertices
  • BeforeManipulatorOperator: avoid "collection was modified" exception

comment:124 Changed 3 years ago by bburlacu

r11262: Added missing Storable constructor to GenealogyGraphNode<T> class. Prefer cloning vertices in the GenealogyAnalyzer. Add In/Out Degree in node tooltip in the GenealogyGraphChart.

comment:125 Changed 3 years ago by bburlacu

r11271: Added specialized diversity analyzers (phenotypic and bottom-up) and phenotypic similarity calculator.

comment:126 Changed 3 years ago by bburlacu

r11272: Adjusted SolutionVariableName for symbolic data analysis similarity calculators.

comment:127 Changed 3 years ago by bburlacu

r11278: GenealogyAnalyzer.cs: add histogram and heatmap for absolute reproductive success values (how many offspring produced by a parent individual).

comment:128 Changed 3 years ago by bburlacu

r11288: Fix calculation of successful offspring ratios in the GenealogyAnalyzer. Simplify cloning in the BeforeManipulatorOperator.

comment:129 Changed 3 years ago by bburlacu

r11291: Added license files and removed unnecessary code from SymbolicDataAnalysisProblem.cs

comment:130 Changed 3 years ago by bburlacu

r11318: Fixed bug and improved handling of elite individuals in the genealogy analyzer. Fixed minor bug in the tracing code. Other minor cosmetic improvements to the GenealogyGraph and FragmentGraphView.

comment:131 Changed 3 years ago by bburlacu

r11349: Merged trunk changes for HeuristicLab.Problems.DataAnalysis.Symbolic.Views.

comment:132 Changed 3 years ago by bburlacu

r11350: Small fix to fragment tracing.

comment:133 Changed 3 years ago by bburlacu

r11383: BeforeManipulatorOperator: preserve fragment data after vertex cloning

comment:134 Changed 3 years ago by bburlacu

r11385: Merged changes to SymbolicExpressionTreeMatching.cs from trunk.

comment:135 Changed 3 years ago by bburlacu

r11386: Extended IGenealogyGraph<T> interface to provide GetByContent and GetById methods. Added implementation in GenealogyGraph<T>.

comment:136 Changed 3 years ago by bburlacu

r11387: Simplified fragment tracing code in the before/after manipulation operators.

comment:137 Changed 3 years ago by bburlacu

r11388: SymbolicDataAnalysisExpressionTracing: added more detailed comments; enforced a safer behaviour in the case of mutation when the traced subtree is contained inside the mutated tree fragment.

comment:138 Changed 3 years ago by bburlacu

r11390: Simplified code in the GenealogyAnalyzer and removed useless property in BeforeCrossoverOperator.cs. Small cosmetic changes in GenealogyAnalyzer.cs and IGenealogyGraph.cs.

comment:139 Changed 3 years ago by bburlacu

r11458: HeuristicLab.Problems.DataAnalysis.Symbolic:

  • Merged trunk changes (SymbolicExpressionImporter)
  • Added Passthrough symbol (does not perform an operation, just transfers the input), adjusted interpreter and opcodes accordingly
  • Added diversity preserving crossover
  • Replaced IDistanceCalculator interface with ISymbolicDataAnalysisExpressionSimilarityCalculator and adjusted similarity calculators
  • Refactored tracing, moved functionality to the TraceCalculator (when this is complete the old code will be removed)

comment:140 Changed 3 years ago by bburlacu

r11459: GenealogyGraph: fix serialization GenealogyGraphNode: avoid boxing/unboxing with the IEnumerable.Cast<T>

comment:141 Changed 3 years ago by bburlacu

r11460: HeuristicLab.Encodings.SymbolicExpressionTreeEncoding:

  • SymbolicExpressionTreeLinearCompiler: add public Compile method accepting an ISymbolicExpressionTreeNode
  • SymbolicExpressionTreeStringFormatter: add public Format method accepting an ISymbolicExpressionTreeNode

comment:142 Changed 3 years ago by bburlacu

r11461: Implemented poly-10 building blocks analyzer and useful genes analyzer (acts as a sort of "intelligent" pruning method).

comment:143 Changed 3 years ago by bburlacu

r11462: Fixed bug and improved the SymbolicDataAnalysisUsefulGenesAnalyzer. Fixed very minor typo in the SymbolicDataAnalysisPoly10Analyzer.

comment:144 Changed 3 years ago by bburlacu

r11473: Finished the TraceCalculator, added tracing for mutation operations.

comment:145 Changed 3 years ago by bburlacu

r11475: Small improvement to GenealogyGraph.cs

comment:146 Changed 3 years ago by bburlacu

r11476: Small changes and improvements to TraceCalculator.cs

comment:147 Changed 3 years ago by bburlacu

r11482: Added the ability to ignore variable weights or constant values to the BottomUpSimilarityCalculator and added parameters to the SymbolicDataAnalysisBottomUpDiversityAnalyzer. Added separate methods in the MaxCommonSubtreeSimilarityCalculator for performing matching with full subtrees (including all leaves) or without (like the old - and better - behavior).

comment:148 Changed 3 years ago by bburlacu

r11488: Merged changes from the BottomUpSimilarityCalculator branch.

comment:149 Changed 3 years ago by bburlacu

r11493: Improved the way the TraceCalculator handles mutations, worked on the SymbolicDataAnalysisPoly10Analyzer (analyzer that tries to identify building blocks for the Poly-10 problem by doing a semantic comparison).

comment:150 Changed 3 years ago by bburlacu

r11499: Merged trunk changes.

comment:151 Changed 3 years ago by bburlacu

r11502: Small refactoring in the GenealogyGraph.

comment:152 Changed 3 years ago by bburlacu

r11503: Added handling of double click event in the genealogy graph (opens a new view of the selected symbolic expression tree). Added highlighting of trace data. Very minor cosmetic changes to the TraceCalculator. Removed resx file from project.

comment:153 Changed 3 years ago by bburlacu

r11506: Updated views (fixed position and anchors).

comment:154 Changed 3 years ago by bburlacu

r11638: Merged trunk changes. Updated PhenotypicSimilarityCalculator, updated FragmentGraphView.

comment:155 Changed 3 years ago by bburlacu

r11639: Migrated the tracking plugins to .NET 4.5, small changes to the GenealogyAnalyzer, added cloning constructor to the GenealogyGraph<T> class.

comment:156 Changed 3 years ago by bburlacu

r11694: Fixed cloning in tracking operators for symbolic data analysis. Added unit tests (in progress).

comment:157 Changed 3 years ago by bburlacu

r11726: Fixed a bug where the fragment root would be set to null (the Difference method returned a null value when the compared subtrees were identical).

comment:158 Changed 3 years ago by bburlacu

r11750: Changed Ancestors and Descendants methods to not include the current graph node as it better fits the definition of what an ancestor (or descendant) is.

comment:159 Changed 3 years ago by bburlacu

r11751: TraceCalculator: Added static TraceSubtree method, renamed private Trace method to TraceRecursive to avoid method name confusion.

comment:160 Changed 3 years ago by bburlacu

r11752: GenealogyGraph: minor refactoring, added a fix/hack to restore correct arcs order for each vertex after deserialization.

comment:161 Changed 3 years ago by bburlacu

r11769: Changed the GenealogyAnalyzer according to the changes in IGenealogyGraph.

comment:162 Changed 3 years ago by bburlacu

r11817: Refactored genealogy graph views. Added an additional class specializing GenealogyGraphView<ISymbolicExpressionTree> for the sole purpose of making the SymbolicDataAnalysisGenealogyGraphView work with the designer. Added property to expose the SymbolicExpressionTreeChart in the GraphicalSymbolicExpressionTreeView in order to be able to wire mouse click events.

comment:163 Changed 3 years ago by bburlacu

r11852: Added some useful functionality to the GenealogyGraphView.

comment:164 Changed 3 years ago by bburlacu

r11855: Improved TraceCalculator performance by caching node lists (2.5-3x speed increase). Removed old tracing code. Insignificant change to the SymbolicDataAnalysisExpressionBeforeManipulatorOperator (rephrased a comment).

comment:165 Changed 3 years ago by bburlacu

r11858: Micro optimization inside the TraceCalculator and inside the GenealogyGraphNode (Children and Parents properties), resulting in maybe 25% speed gain. Corrected item name for the BeforeCrossoverOperator, very minor refactoring inside the BeforeManipulatorOperator. Updated project file.

comment:166 Changed 3 years ago by bburlacu

r11864: Improved functionality of the SymbolicDataAnalysisGenealogyGraphView in terms of graph navigation and display of useful information. Fixed save of tree image to file in the SymbolicExpressionTreeChart.

comment:167 Changed 3 years ago by bburlacu

r11866: Introduced TraceData class in order to make the semantics clear from the code. Changed arcs orientation in the trace graph to match the orientation in the genealogy graph.

comment:168 Changed 3 years ago by bburlacu

r11881:

  • TraceGraph: Improved saving of trace data, changed GetTraceNode method to AddTraceNode and made the code more clear.
  • SymbolicDataAnalysisGenealogyGraphView: added highlighting of trace information (when visualizing trace graphs)
  • GenealogyGraphChart: removed useless code

comment:169 Changed 3 years ago by bburlacu

r11891: Deleted old files that were no longer needed.

comment:170 Changed 3 years ago by bburlacu

r11925:

  • Removed unused code
  • Adapted genealogy graph according to the trunk changes to the directed graph
  • Adapted trace calculator and before/after operators
  • Introduced a CloneableCollection to help save node lists

comment:171 Changed 3 years ago by bburlacu

r11926: Merged trunk changes

comment:172 Changed 3 years ago by bburlacu

r11927: Fixed compilation errors and removed leftover similarity calculators, re-added ISymbolicExpressionTreeNodeSimilarityComparer interface and PhenotypicSimilarityCalculator.

comment:173 Changed 3 years ago by bburlacu

r11928: Added missing clone method to Fragment<T>, changed storable constructor access modifier to protected.

comment:174 Changed 3 years ago by bburlacu

r11965: Merged trunk changes and updated the SymbolicDataAnalysisBottomUpDiversityAnalyzer.

comment:175 Changed 3 years ago by bburlacu

r11968: Small performance tweaks to the TraceCalculator. Reverted SubtreeCrossover to old behavior: do not clone subtrees (as it breaks tracing).

comment:176 Changed 3 years ago by bburlacu

r11969: Restore correct order of arcs when cloning or deserializing a GenealogyGraph (due to changes in the trunk in the DirectedGraph, previous code was not working anymore).

comment:177 Changed 3 years ago by bburlacu

r11979: Improve performance of the TraceCalculator by marking already visited trace nodes in order to avoid duplicate effort.

comment:178 Changed 3 years ago by bburlacu

r12017:

  • Added missing cloning constructors to the phenotypic and bottom-up similarity analyzers
  • Added missing cloning of private members in the PhenotypicSimilarityCalculator cloning constructor
  • Merged trunk changes into the SymbolicExpressionTreeBottomUpSimilarityCalculator
  • Minor layout fixes to the SymbolicDataAnalysisGenealogyGraphView

comment:179 Changed 3 years ago by bburlacu

r12155: Merged trunk changes.

comment:180 Changed 3 years ago by bburlacu

r12156: Added base class for genealogy analyzers, added fragment length analyzer and trace overlap analyzer.

comment:181 Changed 3 years ago by bburlacu

r12163: Split EvolutionTrackingAnalyzer into generic and non-generic class. Added SelectionSchemeAnalyzer.

comment:182 Changed 3 years ago by bburlacu

r12208: Merged trunk and worked on the graph view.

comment:183 Changed 3 years ago by bburlacu

r12225: Work in progress for calculating sampling counts for subtrees in the population: updated TraceCalculator to aggregate tracing statistics, updated SymbolicDataAnalysisGenealogyGraphView, added SymbolicDataAnalysisSubtreeSampleCountAnalyzer.

comment:184 Changed 3 years ago by bburlacu

r12231: Changed the way node highlighting is performed (taking into account sampling count relative to current generation). Made NodeWeight field storable in the SymbolicExpressionTreeNode. Updated the statistics counting in the TraceCalculator.

comment:185 Changed 2 years ago by bburlacu

r12265: Some more work on subtree sample counting.

comment:186 Changed 2 years ago by bburlacu

r12283: Updated genealogy graph view and worked on subtree sample count.

comment:187 Changed 2 years ago by bburlacu

r12287: Added storable attributes to the before/after evolution tracking operators in HeuristicLab.Problems.DataAnalysis.Symbolic. Very small changes to the trace calculator (updated license header, removed old sample count code).

comment:188 Changed 2 years ago by bburlacu

r12318: Fixed small bug/typo in the SymbolicDataAnalysisPoly10Analyzer

comment:189 Changed 2 years ago by bburlacu

r12406: Removed obsolete FrequentFragmentsDialog, added a checkbox to select the subtree matching mode in the SymbolicDataAnalysisGenealogyGraphView, removed commented code from the SymbolicDataAnalysisPoly10Analyzer.

comment:190 Changed 2 years ago by bburlacu

r12891: Merge trunk changes. Remove dead code from the genealogy analyzer.

comment:191 Changed 2 years ago by bburlacu

r12892: Made GenealogyAnalyzer class abstract and added abstract method EvaluateIntermediateChildren in order to assign qualities to intermediate vertices in the genealogy graph. Implemented SymbolicDataAnalysisGeneticOperatorImprovementAnalyzer which shows parent-child quality statistics for crossover and mutation.

comment:192 Changed 2 years ago by bburlacu

r12894: Small updates to SelectionSchemeAnalyzer and SymbolicDataAnalysisGeneticOperatorImprovementAnalyzer.

comment:193 Changed 2 years ago by bburlacu

r12929: Initial implementation of tree pattern matching algorithm by Götz et al. (​http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.182.5440). Added wildcard symbols and nodes and updated the matching code accordingly.

comment:194 Changed 2 years ago by bburlacu

r12933: Added a parameter to the tree query matching algorithm to specify whether only strict parent-child pairs in the data tree should be matched with the query (otherwise ancestor-descendant pairs are also considered valid).

comment:195 Changed 2 years ago by bburlacu

r12941: Refactored the tree query match code and removed debugging code (as the same information can be obtained with IntelliTrace). Added also level restriction to node matching in order to be more strict regarding schema matching.

comment:196 Changed 2 years ago by bburlacu

r12942: Forgot to remove unused parameter.

Last edited 2 years ago by bburlacu (previous) (diff)

comment:197 Changed 2 years ago by bburlacu

r12951:

  • Slight refactor in QueryMatch.cs
  • Added a parameter to the genealogy analyzer for removing older generations from the graph (useful to conserve memory in experiments)
  • Updated wildcard nodes (added persistence & cloning)
  • Implemented diversification strategy based on schema frequencies & phenotypic similarity as a separate operator (for now keep also the analyzer)
  • Updated license headers
  • Added QueryMatch performance test (to be expanded)

comment:198 Changed 2 years ago by bburlacu

r12952: Add tracking of the number of trees that are mutated during the diversification phase by the SchemaEvaluator.

comment:199 Changed 2 years ago by bburlacu

r12958: Properly remove older generations in the genealogy graph. Fix namespaces in the schema diversification operators.

comment:200 Changed 2 years ago by bburlacu

r12965: Fix memory leak in SchemaCleanupOperator

comment:201 Changed 2 years ago by bburlacu

r12966: Fixed unit test errors.

comment:202 Changed 2 years ago by bburlacu

r12970: Fix plugin dependencies. Small changes to the SchemaEvaluator: require a minimum of 2 matches for each schema (regardless of the MinimumSchemaFrequency) and use Math.Floor instead of Math.Round when determining the number of individuals to be replaced according to the ReplacementRatio.

comment:203 Changed 2 years ago by bburlacu

r12979:

  • added parameters to the SchemaCreator for calculating things in parallel
  • added parallel code to the SchemaEvaluator
  • implemented quality calculation and saving of estimated values in the scope in the UpdateEstimatedValuesOperator
  • small refactoring of QueryMatch (added some useful methods and made NodeInfo internal)
  • updated query match unit test

comment:204 Changed 2 years ago by bburlacu

r12988: Performance improvements:

  • QueryMatch.cs: eliminate unnecessary ToList() call and expensive GetBranchLevel calls
  • Diversification: eliminated creation of shallow copies of the individual subscopes as it was either too slow (due to events being registered/deregistered when variables are added to the scope) or too leaky (if attempting to clear the scopes without clearing the variables then the code is leaking EventHandlers)
  • Aggregated diversification statistics separately with the help of some parameters set up in the SchemaCreator and SchemaEvaluator
  • Made code in the UpdateEstimatedValuesOperator perform exactly as in the evaluator (updating quality and estimated values)
  • Removed no longer needed SchemaCleanupOperator
  • Do not evaluate intermediate vertices in the genealogy analyzer if the TrimOlderGenerations flag is activated

New functionality:

  • parameter to control the fraction of the population to be considered by the diversification strategy
  • parameter to control whether individuals may be matched by a schema and mutated only once (exclusive matching)
  • parameter to control whether linear scaling should be applied to the estimated values used for the calculation of phenotypic similarity (default: yes)

comment:205 Changed 2 years ago by bburlacu

r13061: Adapted visualization code according to the changes in the HeuristicLab.Visualization branch.

comment:206 Changed 22 months ago by bburlacu

r13399: Fix compilation errors caused by changes in the Visualization branch.

comment:207 Changed 22 months ago by bburlacu

r13424: Fixed bug due to incorrect caching in the traceMap where in some cases trace graph arcs could contain incorrect trace data (this does not affect or invalidate any previous results as arc information is only being used for visualization). Simplified code by removing the traceMap completely and passing the indices directly to the trace method.

comment:208 Changed 22 months ago by bburlacu

r13426: Updated the GenealogyGraphChart using new chart modes from HeuristicLab.Visualization

comment:209 Changed 21 months ago by bburlacu

r13479: Improved caching in TraceCalculator

comment:210 Changed 21 months ago by bburlacu

r13480: Schema diversification strategy: implement adaptive replacement ratio and added number of evaluated solutions per generation to the diversification statistics operator

comment:211 Changed 21 months ago by bburlacu

r13481: Added schema frequency analyzer, removed unused code from the SymbolicDataAnalysisSubtreeSampleCountAnalyzer

comment:212 Changed 21 months ago by bburlacu

r13482: Merged trunk changes

comment:213 Changed 21 months ago by bburlacu

r13495: Improve the average operator improvement and average fragment length analyzers. Update genealogy analyzer to always evaluate the intermediate children.

comment:214 Changed 21 months ago by bburlacu

r13496: Update diversification operators (add extra evaluations to the evaluation counter, add the option for strict schema matching, add additional replacement ratio update rules.

comment:215 Changed 21 months ago by bburlacu

r13519: Fix cloning of CloneableCollection to avoid a stack overflow exception in some cases.

comment:216 Changed 20 months ago by bburlacu

r13527: Fixed the way operator improvement is calculated so it also works when a diversification strategy is applied (introducing multiple intermediate vertices between parent and child). Minor code refactoring in the diversification operators. Fixed very small typo in the BeforeManipulatorOperator.

comment:217 Changed 19 months ago by bburlacu

r13624: Improve the SchemaFrequencyAnalyzer and add calculation of schema genotype and phenotype diversities.

comment:218 Changed 19 months ago by bburlacu

r13625: Small refactor of the SchemaCreator to avoid generating duplicate schemas when non-strict matching is used.

comment:219 Changed 18 months ago by bburlacu

r13747: Extend the fragment length analyzer to calculate fragment size distributions for both the swapped and replaced fragments.

comment:220 Changed 16 months ago by bburlacu

r13875: Merge trunk changes.

comment:221 Changed 16 months ago by bburlacu

r13876:

  • SchemaCreator: Replace cutpoints with wildcards from the bottom up when generating schemas.
  • Add temporary workaround to restore parent links in child nodes if they become corrupted.

comment:222 Changed 16 months ago by bburlacu

r13877: Improve SymbolicDataAnalysisGenealogyGraphView by using a better coloring scheme for frequent subtrees. Fix bug when reaching the end of the genealogy. Minor refactorings.

comment:223 Changed 16 months ago by bburlacu

r13892: Fix bug in tree matching with wildcards (used by the genealogy graph view instead of the query match) and improved subtree selection and matching functionality in the genealogy graph view.

comment:224 Changed 15 months ago by bburlacu

r13944: Fix bug in fragment length calculator leading to exception when we try to compute the average out of an empty sequence of fragment lengths.

comment:225 Changed 12 months ago by bburlacu

r14312: Merge trunk changes. Delete unnecessary files (sliding window).

comment:226 Changed 12 months ago by bburlacu

r14326: Refactor QueryMatch class to make the code more readable and easier to understand.

comment:227 Changed 10 months ago by bburlacu

r14426: QueryMatch: implement more elaborate node comparison (better accuracy when matching schemas).

comment:228 Changed 10 months ago by bburlacu

r14427: Extract common methods (used by the schema creator and the schema frequency analyzer) in static SchemaUtil class. Make AnyNode constructor public.

comment:229 Changed 8 months ago by bburlacu

r14574: Fix data table cloning when storing history in the SymbolicDataAnalysisFragmentLengthAnalyzer. Fix some typos and formatting.

comment:230 Changed 8 months ago by bburlacu

r14575: Minor refactoring in the TraceCalculator (use Tuple.Create factory method to make the code more readable).

comment:231 Changed 8 months ago by bburlacu

r14576: Merge trunk changes.

comment:232 Changed 8 months ago by bburlacu

r14626: Exclude intermediate individuals from schema matching in the schema frequency analyzer.

Last edited 8 months ago by bburlacu (previous) (diff)

comment:233 Changed 8 months ago by bburlacu

r14627: Add option when generating schemas to only consider parents that have produced a minimum number of offspring.

comment:234 Changed 8 months ago by bburlacu

r14624: QueryMatch.cs: implement caching of NodeInfo lists to improve performance when repeatedly matching trees against schemas.

comment:235 Changed 5 months ago by bburlacu

r14878: Merge trunk changes.

comment:236 follow-up: Changed 5 months ago by gkronber

Priority highest? How do we proceed with this ticket?

comment:237 in reply to: ↑ 236 Changed 5 months ago by bburlacu

Replying to gkronber:

Priority highest? How do we proceed with this ticket?

The plan is to merge this functionality back into the trunk, and probably to provide a separate type of GA that also performs "tracking".

comment:238 Changed 2 weeks ago by bburlacu

r15351: Merge trunk changes, add a couple of convenience parameters to the SchemaCreator.

Note: See TracTickets for help on using tickets.