Opened 7 years ago
Closed 4 years ago
#1081 closed feature request (done)
Plugin for symbolic time series prognosis with GP
Reported by: | gkronber | Owned by: | gkronber |
---|---|---|---|
Priority: | medium | Milestone: | HeuristicLab 3.3.8 |
Component: | Problems.DataAnalysis | Version: | 3.3.8 |
Keywords: | Cc: |
Description
Change History (105)
comment:1 Changed 7 years ago by gkronber
- Status changed from new to assigned
comment:2 Changed 7 years ago by gkronber
comment:3 Changed 7 years ago by gkronber
removed commented code with r4023.
comment:4 Changed 7 years ago by gkronber
Moved code for calculation of covariance from the scaled MSE evaluator into a separate online evaluator with r4027.
comment:5 Changed 7 years ago by mkommend
Removed resource file for LaggedVariableView r4029.
comment:6 Changed 7 years ago by gkronber
- Version changed from 3.3 to 3.3.1
comment:7 Changed 7 years ago by gkronber
Added plugin for time series prognosis with r4113.
comment:8 Changed 7 years ago by gkronber
set svn:ignore properties with r4114.
comment:9 Changed 7 years ago by gkronber
- Version changed from 3.3.1 to branch
comment:10 Changed 7 years ago by gkronber
- Milestone changed from HeuristicLab x.x.x to HeuristicLab 3.3.3
comment:11 Changed 6 years ago by gkronber
- Milestone changed from HeuristicLab 3.3.3 to HeuristicLab x.x.x
comment:12 Changed 6 years ago by gkronber
- Component changed from ZZZ OBSOLETE: Problems.DataAnalysis.MultiVariate.TimeSeriesPrognosis to Problems.DataAnalysis
- Milestone changed from HeuristicLab 3.3.x Backlog to HeuristicLab 3.3.6
- Status changed from assigned to accepted
- Version changed from branch to 3.3.5
comment:13 Changed 6 years ago by gkronber
r6802: added classes (problem, evaluators, analyzers, solution, model, online-calculators, and views) for time series prognosis problems and added an algorithm implementation to generation linear AR (auto-regressive) time series prognosis solution.
comment:14 Changed 6 years ago by gkronber
r6807: added views for time series prognosis
comment:15 Changed 6 years ago by gkronber
r6808: fixed build
comment:16 Changed 6 years ago by gkronber
- added configuration of default grammar for time-series prognosis
- improved multiple update of variable impacts result
comment:17 Changed 6 years ago by gkronber
- Owner changed from gkronber to mkommend
- Status changed from accepted to reviewing
comment:18 Changed 6 years ago by gkronber
- Owner changed from mkommend to gkronber
- Status changed from reviewing to assigned
comment:19 Changed 6 years ago by gkronber
- Status changed from assigned to accepted
comment:20 Changed 6 years ago by gkronber
- cleared up definition of accuracy metrics for time series prognosis to make the distinction between one n-step forecast and n one-step forecasts clearer
- implemented calculators for time series accuracy metrics to support the calculation of the average accuracy over m n-step forecasts,
- and adapted the unit tests accordingly.
comment:21 Changed 6 years ago by gkronber
Open tasks:
- implement unit tests for n-step forecasts (only one-step forecasts are tested right now)
- write blog post / wiki page to document the accuracy metrics
- add a benchmark test set for time series prognosis with financial data
comment:22 Changed 6 years ago by gkronber
r7058: added unit tests for directional symmetry and Theil's U and multi-step forecasts
comment:23 Changed 6 years ago by gkronber
A larger overhaul of the time series implementation is necessary.
The most important aspect of time series analysis is prognosis of future values. As of now we only support one-step forecasts from y(t) to y(t+1). However, often it is necessary to create a forecast for the next n time-steps. For uni-variate time series, it would be possible to calculate the forecast outside of HeuristicLab, by simple recurrence (using the first one-step prognosis y(t+1) as additional input for the next one-step prognosis y(t+2) and so on). It would be nice to support this also directly in HeuristicLab and show for instance the line chart of the forecast for the whole test partition.
The second problem when n-step forecasts are supported arises when additional variables are used as inputs for the time-series model. Because the values of the input variables are usually not known for the forecast period. This issue can be solved by learning a multi-variate time-series model instead.
comment:24 Changed 6 years ago by gkronber
Because of the considerations mentioned above I decided to pull the time-series plugins from the 3.3.6 release as this would lead to difficulties when we have to maintain compatibility in future releases.
The time-series prognosis functionality will be released with 3.3.7.
comment:25 Changed 6 years ago by gkronber
comment:26 Changed 6 years ago by gkronber
r7098: created a branch for the development of time series prognosis features.
comment:27 Changed 6 years ago by gkronber
- Milestone changed from HeuristicLab 3.3.6 to HeuristicLab 3.3.7
comment:28 Changed 6 years ago by gkronber
comment:29 Changed 6 years ago by gkronber
r7100: worked on multi-variate time series prognosis
comment:30 Changed 6 years ago by gkronber
r7120: implemented multi-variate symbolic expression tree interpreter for time series prognosis.
comment:31 Changed 6 years ago by gkronber
r7129: worked on multi-variate time series prognosis
comment:32 Changed 6 years ago by gkronber
r7154: worked on multi-variate time series prognosis
comment:33 Changed 6 years ago by gkronber
r7160: worked on multi-variate time series prognosis
comment:34 Changed 6 years ago by gkronber
- implemented remaining metrics for time series prognosis solutions
- added estimation limits
- fixed training and validation best solution analyzers
- implemented overfitting analyzer.
comment:35 Changed 6 years ago by gkronber
r7184: removed resx files
comment:36 Changed 6 years ago by gkronber
r7194: small changes in calculation of time series solution results
comment:37 Changed 6 years ago by gkronber
r7213 merged r7103:7209 from trunk into time series branch
comment:38 Changed 5 years ago by gkronber
r7268: merged r7214:7266 from trunk into time series branch.
comment:39 Changed 5 years ago by gkronber
r7460: merged r7266:7459 from the trunk into the time series prognosis branch.
comment:40 Changed 5 years ago by gkronber
r7461: fixed a few problems related to a previous incorrect merge and removed unrelated projects from the solution.
comment:41 Changed 5 years ago by gkronber
In the simplified for time series solutions the correct horizon should be used to calculate impacts.
comment:42 Changed 5 years ago by gkronber
r7463: improved formatter, line chart view, simplifier for time series prognosis solutions
comment:43 Changed 5 years ago by gkronber
- Milestone changed from HeuristicLab 3.3.7 to HeuristicLab 3.3.x Backlog
comment:44 Changed 5 years ago by gkronber
r7615: merged r7462:7609 from trunk into time series branch
comment:45 Changed 5 years ago by gkronber
r7842: merged r7609:7840 from trunk into time series branch
comment:46 Changed 5 years ago by gkronber
r7843: fixed compile errors after merging changes from the trunk
comment:47 Changed 5 years ago by mkommend
r7886: Adapted timeseries branch to contain only modified plugins.
comment:48 Changed 5 years ago by mkommend
r7889: Branched problem instances to enable CSV import for timeseries problems.
comment:49 Changed 5 years ago by mkommend
- Owner changed from gkronber to mkommend
- Status changed from accepted to assigned
comment:50 Changed 5 years ago by mkommend
- Status changed from assigned to accepted
comment:51 Changed 5 years ago by mkommend
r7890: Implemented CSV problems instances provider for time series prognosis problems.
comment:52 Changed 5 years ago by mkommend
r7926: Added interpreter performance unit test for type coherent grammar.
comment:53 Changed 5 years ago by mkommend
r7930: Refactored symbolic expression tree interpreter in preparation for autoregressive single variate prognosis.
comment:54 Changed 5 years ago by gkronber
- Version changed from 3.3.5 to branch
comment:55 Changed 5 years ago by mkommend
r7989: Improved performance of time series prognosis.
comment:56 Changed 5 years ago by mkommend
r7991: Updated estimation limits correctly in TimeSeriesPrognosisProblem.
comment:57 Changed 5 years ago by mkommend
r7998: Corrected time series interpreter and updated visualizations.
comment:58 Changed 5 years ago by mkommend
r8010: Corrected time series solution results and implemented new models.
comment:59 Changed 5 years ago by mkommend
- Milestone changed from HeuristicLab 3.3.x Backlog to HeuristicLab 3.3.8
r8114: Corrected scaling behavior of symbolic time series models.
comment:60 Changed 5 years ago by mkommend
r8116: Branched HL.DataAnalysis to add AR models.
comment:61 Changed 5 years ago by mkommend
r8428: Temporary removed files in branch to prepare for trunk update.
comment:62 Changed 5 years ago by mkommend
r8430: Intermediate commit of trunk updates - interpreter changes must be redone.
comment:63 Changed 5 years ago by mkommend
r8431: Moved interpreter specific classes to separate directory.
comment:64 Changed 5 years ago by mkommend
r8432: Corrected path of previous move operation.
comment:65 Changed 5 years ago by mkommend
r8436: Merged changes from trunk into timeseries branch and refactored GP interpreter.
comment:66 Changed 5 years ago by mkommend
r8458: Derived time series classes from regression classes to avoid code duplication.
comment:67 Changed 5 years ago by mkommend
r8459: Code clean up in time series prognosis model and problem data.
comment:68 Changed 5 years ago by mkommend
r8460: Added horizon parameters to timeseries problem data.
comment:69 Changed 5 years ago by mkommend
r8468: Adapated calculation of results for horizons larger than 1.
comment:70 Changed 5 years ago by mkommend
- Added autoregressive target variable Symbol
- Merged trunk changes into the branch.
comment:71 Changed 5 years ago by mkommend
r8483: Corrected evaluators and time series models.
comment:72 Changed 5 years ago by mkommend
r8487: Added ErrorCharacteristics View for time series models.
comment:73 Changed 5 years ago by mkommend
r8490: Updated TimeSeriesPrognosisSolutionView.
comment:74 Changed 5 years ago by mkommend
r8742: Merged trunk changes and fixed compilation errors due to the merge.
comment:75 Changed 5 years ago by mkommend
r8743: Readded ErrorCharacteristicsView for time series prognosis solutions.
comment:76 Changed 5 years ago by mkommend
r8750: Extracted prognosis results into separate class and added a view for them.
comment:77 Changed 5 years ago by mkommend
r8762: Fixed serialization and corrected AR(n) modeling.
comment:78 Changed 5 years ago by mkommend
r8764: Removed wrong MA models from timeseries prognosis branch.
comment:79 Changed 5 years ago by mkommend
r8765: Fixed minor bugs in time series prognosis views.
comment:80 Changed 5 years ago by mkommend
r8766: Reintegrated trunk updates in timeseries branch.
comment:81 Changed 5 years ago by mkommend
r8767: Merged remaining changes from the trunk.
comment:82 Changed 5 years ago by mkommend
r8768: Corrected plugin dependencies.
comment:83 Changed 5 years ago by mkommend
r8770: Corrected plugin.cs.frame for symbolic time series prognosis views.
comment:84 Changed 5 years ago by mkommend
r8789: Merged trunk changes into timeseries branch.
comment:85 Changed 5 years ago by mkommend
- Owner changed from mkommend to gkronber
- Status changed from accepted to reviewing
r8798: Reintegrated time series modeling branch into trunk.
comment:86 Changed 5 years ago by mkommend
- Owner changed from gkronber to mkommend
- Status changed from reviewing to assigned
comment:87 Changed 5 years ago by mkommend
r8799: Corrected time series projects (copy local, output path).
comment:88 Changed 5 years ago by mkommend
r8801: Changed project reference to an assembly reference in Problems.DataAnalysis.Symbolic.TimeSeriesPrognosis and updated build dependencies in the solution file.
This change was necessary to allow compilation on the build server.
comment:89 Changed 5 years ago by mkommend
- Owner changed from mkommend to gkronber
- Status changed from assigned to reviewing
comment:90 Changed 5 years ago by abeham
r8851: Changed assembly reference to a project reference
It always gave me build errors when starting from a fresh checkout. I don't know why it didn't fail on the build server, but it should have.
comment:91 Changed 5 years ago by gkronber
comment:92 Changed 5 years ago by abeham
Ok, the problem that I was seeing seems to be related to this bug in msbuild. The article says it should be fixed in .Net 4.5.
comment:93 Changed 5 years ago by mkommend
r8940: Added EstimationLimits to ISymbolicTimeSeriesPrognosisModel.
comment:94 Changed 5 years ago by gkronber
Again had to correct revision numbers in previous comments. Please be careful.
comment:95 follow-ups: ↓ 101 ↓ 102 Changed 5 years ago by gkronber
review comments:
fields in TimeSeriesPrognosisResults are not cloned.Update of EvaluatedSolutions is not thread safe in the interpreters.- Cannot activate the symbol LaggedVariable in the GUI.
- the default grammar does only allow real-valued symbols in the root (not time series symbols)
- node impact calculation should be based on the currently selected evaluator. In particular the currently selected training horizon value should be used. Right now impacts are calculated in the same way as for regression models.
- It would be nice to also provide a line-chart showing the n-step forecast.
comment:96 Changed 5 years ago by gkronber
- locking for update of EvaluatedSolutions in interpreter
- cloning of fields in TimeSeriesPrognosisResults
comment:97 Changed 4 years ago by gkronber
I think the training partition for time-series instances should automatically start at 1 or 10 otherwise this has to be changed manually each time (otherwise an exception occurs)
comment:98 Changed 4 years ago by gkronber
r9452: created an override for the Load() method in SymbolicTimeSeriesPrognosisSingleObjectiveProblem to set the first index of the start partition to 10 (prevents ArgumentException)
comment:99 Changed 4 years ago by gkronber
- Owner changed from gkronber to mkommend
- Status changed from reviewing to assigned
- Version changed from branch to 3.3.7
comment:100 Changed 4 years ago by gkronber
Please fix the grammar/symbol problems mentioned in comment:95 for the 3.3.8 release. The other issues should be tracked in new tickets.
comment:101 in reply to: ↑ 95 Changed 4 years ago by mkommend
r9459: Updated TypeCoherentGrammar to take time-series, power, and condition symbols into account.
comment:102 in reply to: ↑ 95 Changed 4 years ago by mkommend
- Owner changed from mkommend to gkronber
- Status changed from assigned to reviewing
review comments:
- Cannot activate the symbol LaggedVariable in the GUI.
All variable symbols can only be activated if at least one input variable is allowed.
- the default grammar does only allow real-valued symbols in the root (not time series symbols)
This was corrected with r9459. However, I have not updated the special functions.
comment:103 Changed 4 years ago by mkommend
comment:104 Changed 4 years ago by gkronber
- Status changed from reviewing to readytorelease
comment:105 Changed 4 years ago by swagner
- Resolution set to done
- Status changed from readytorelease to closed
- Version changed from 3.3.7 to 3.3.8
Worked on symbolic regression classes to prepare for time series prognosis plugin with r4022.