Free cookie consent management tool by TermsFeed Policy Generator

Opened 14 years ago

Closed 11 years ago

#1081 closed feature request (done)

Plugin for symbolic time series prognosis with GP

Reported by: gkronber Owned by: gkronber
Priority: medium Milestone: HeuristicLab 3.3.8
Component: Problems.DataAnalysis Version: 3.3.8
Keywords: Cc:

Description


Change History (105)

comment:1 Changed 14 years ago by gkronber

  • Status changed from new to assigned

comment:2 Changed 14 years ago by gkronber

Worked on symbolic regression classes to prepare for time series prognosis plugin with r4022.

comment:3 Changed 14 years ago by gkronber

removed commented code with r4023.

comment:4 Changed 14 years ago by gkronber

Moved code for calculation of covariance from the scaled MSE evaluator into a separate online evaluator with r4027.

comment:5 Changed 14 years ago by mkommend

Removed resource file for LaggedVariableView r4029.

comment:6 Changed 14 years ago by gkronber

  • Version changed from 3.3 to 3.3.1

comment:7 Changed 14 years ago by gkronber

Added plugin for time series prognosis with r4113.

comment:8 Changed 14 years ago by gkronber

set svn:ignore properties with r4114.

comment:9 Changed 14 years ago by gkronber

  • Version changed from 3.3.1 to branch

comment:10 Changed 13 years ago by gkronber

  • Milestone changed from HeuristicLab x.x.x to HeuristicLab 3.3.3

comment:11 Changed 13 years ago by gkronber

  • Milestone changed from HeuristicLab 3.3.3 to HeuristicLab x.x.x

comment:12 Changed 13 years ago by gkronber

  • Component changed from ZZZ OBSOLETE: Problems.DataAnalysis.MultiVariate.TimeSeriesPrognosis to Problems.DataAnalysis
  • Milestone changed from HeuristicLab 3.3.x Backlog to HeuristicLab 3.3.6
  • Status changed from assigned to accepted
  • Version changed from branch to 3.3.5

comment:13 Changed 13 years ago by gkronber

r6802: added classes (problem, evaluators, analyzers, solution, model, online-calculators, and views) for time series prognosis problems and added an algorithm implementation to generation linear AR (auto-regressive) time series prognosis solution.

comment:14 Changed 13 years ago by gkronber

r6807: added views for time series prognosis

comment:15 Changed 13 years ago by gkronber

r6808: fixed build

comment:16 Changed 13 years ago by gkronber

r6811:

  • added configuration of default grammar for time-series prognosis
  • improved multiple update of variable impacts result

comment:17 Changed 13 years ago by gkronber

  • Owner changed from gkronber to mkommend
  • Status changed from accepted to reviewing

comment:18 Changed 12 years ago by gkronber

  • Owner changed from mkommend to gkronber
  • Status changed from reviewing to assigned

comment:19 Changed 12 years ago by gkronber

  • Status changed from assigned to accepted

comment:20 Changed 12 years ago by gkronber

r6974:

  • cleared up definition of accuracy metrics for time series prognosis to make the distinction between one n-step forecast and n one-step forecasts clearer
  • implemented calculators for time series accuracy metrics to support the calculation of the average accuracy over m n-step forecasts,
  • and adapted the unit tests accordingly.

comment:21 Changed 12 years ago by gkronber

Open tasks:

  • implement unit tests for n-step forecasts (only one-step forecasts are tested right now)
  • write blog post / wiki page to document the accuracy metrics
  • add a benchmark test set for time series prognosis with financial data

comment:22 Changed 12 years ago by gkronber

r7058: added unit tests for directional symmetry and Theil's U and multi-step forecasts

comment:23 Changed 12 years ago by gkronber

A larger overhaul of the time series implementation is necessary.

The most important aspect of time series analysis is prognosis of future values. As of now we only support one-step forecasts from y(t) to y(t+1). However, often it is necessary to create a forecast for the next n time-steps. For uni-variate time series, it would be possible to calculate the forecast outside of HeuristicLab, by simple recurrence (using the first one-step prognosis y(t+1) as additional input for the next one-step prognosis y(t+2) and so on). It would be nice to support this also directly in HeuristicLab and show for instance the line chart of the forecast for the whole test partition.

The second problem when n-step forecasts are supported arises when additional variables are used as inputs for the time-series model. Because the values of the input variables are usually not known for the forecast period. This issue can be solved by learning a multi-variate time-series model instead.

comment:24 Changed 12 years ago by gkronber

Because of the considerations mentioned above I decided to pull the time-series plugins from the 3.3.6 release as this would lead to difficulties when we have to maintain compatibility in future releases.

The time-series prognosis functionality will be released with 3.3.7.

comment:25 Changed 12 years ago by gkronber

r7097: reverse merged r6802, r6807:6808, r6811, r6974, r7058 to prepare for the 3.3.6 release

comment:26 Changed 12 years ago by gkronber

r7098: created a branch for the development of time series prognosis features.

comment:27 Changed 12 years ago by gkronber

  • Milestone changed from HeuristicLab 3.3.6 to HeuristicLab 3.3.7

comment:28 Changed 12 years ago by gkronber

r7099: merged old changesets r6802, r6807:6808, r6811, r6974, r7058 from the trunk into the TimeSeries branch to bring it to version r7096.

comment:29 Changed 12 years ago by gkronber

r7100: worked on multi-variate time series prognosis

comment:30 Changed 12 years ago by gkronber

r7120: implemented multi-variate symbolic expression tree interpreter for time series prognosis.

comment:31 Changed 12 years ago by gkronber

r7129: worked on multi-variate time series prognosis

comment:32 Changed 12 years ago by gkronber

r7154: worked on multi-variate time series prognosis

comment:33 Changed 12 years ago by gkronber

r7160: worked on multi-variate time series prognosis

comment:34 Changed 12 years ago by gkronber

r7183:

  • implemented remaining metrics for time series prognosis solutions
  • added estimation limits
  • fixed training and validation best solution analyzers
  • implemented overfitting analyzer.

comment:35 Changed 12 years ago by gkronber

r7184: removed resx files

comment:36 Changed 12 years ago by gkronber

r7194: small changes in calculation of time series solution results

comment:37 Changed 12 years ago by gkronber

r7213 merged r7103:7209 from trunk into time series branch

comment:38 Changed 12 years ago by gkronber

r7268: merged r7214:7266 from trunk into time series branch.

comment:39 Changed 12 years ago by gkronber

r7460: merged r7266:7459 from the trunk into the time series prognosis branch.

comment:40 Changed 12 years ago by gkronber

r7461: fixed a few problems related to a previous incorrect merge and removed unrelated projects from the solution.

comment:41 Changed 12 years ago by gkronber

In the simplified for time series solutions the correct horizon should be used to calculate impacts.

comment:42 Changed 12 years ago by gkronber

r7463: improved formatter, line chart view, simplifier for time series prognosis solutions

comment:43 Changed 12 years ago by gkronber

  • Milestone changed from HeuristicLab 3.3.7 to HeuristicLab 3.3.x Backlog

comment:44 Changed 12 years ago by gkronber

r7615: merged r7462:7609 from trunk into time series branch

comment:45 Changed 12 years ago by gkronber

r7842: merged r7609:7840 from trunk into time series branch

comment:46 Changed 12 years ago by gkronber

r7843: fixed compile errors after merging changes from the trunk

comment:47 Changed 12 years ago by mkommend

r7886: Adapted timeseries branch to contain only modified plugins.

comment:48 Changed 12 years ago by mkommend

r7889: Branched problem instances to enable CSV import for timeseries problems.

comment:49 Changed 12 years ago by mkommend

  • Owner changed from gkronber to mkommend
  • Status changed from accepted to assigned

comment:50 Changed 12 years ago by mkommend

  • Status changed from assigned to accepted

comment:51 Changed 12 years ago by mkommend

r7890: Implemented CSV problems instances provider for time series prognosis problems.

comment:52 Changed 12 years ago by mkommend

r7926: Added interpreter performance unit test for type coherent grammar.

comment:53 Changed 12 years ago by mkommend

r7930: Refactored symbolic expression tree interpreter in preparation for autoregressive single variate prognosis.

comment:54 Changed 12 years ago by gkronber

  • Version changed from 3.3.5 to branch

comment:55 Changed 12 years ago by mkommend

r7989: Improved performance of time series prognosis.

comment:56 Changed 12 years ago by mkommend

r7991: Updated estimation limits correctly in TimeSeriesPrognosisProblem.

comment:57 Changed 12 years ago by mkommend

r7998: Corrected time series interpreter and updated visualizations.

comment:58 Changed 12 years ago by mkommend

r8010: Corrected time series solution results and implemented new models.

comment:59 Changed 12 years ago by mkommend

  • Milestone changed from HeuristicLab 3.3.x Backlog to HeuristicLab 3.3.8

r8114: Corrected scaling behavior of symbolic time series models.

comment:60 Changed 12 years ago by mkommend

r8116: Branched HL.DataAnalysis to add AR models.

comment:61 Changed 12 years ago by mkommend

r8428: Temporary removed files in branch to prepare for trunk update.

comment:62 Changed 12 years ago by mkommend

r8430: Intermediate commit of trunk updates - interpreter changes must be redone.

comment:63 Changed 12 years ago by mkommend

r8431: Moved interpreter specific classes to separate directory.

comment:64 Changed 12 years ago by mkommend

r8432: Corrected path of previous move operation.

comment:65 Changed 12 years ago by mkommend

r8436: Merged changes from trunk into timeseries branch and refactored GP interpreter.

comment:66 Changed 12 years ago by mkommend

r8458: Derived time series classes from regression classes to avoid code duplication.

comment:67 Changed 12 years ago by mkommend

r8459: Code clean up in time series prognosis model and problem data.

comment:68 Changed 12 years ago by mkommend

r8460: Added horizon parameters to timeseries problem data.

comment:69 Changed 12 years ago by mkommend

r8468: Adapated calculation of results for horizons larger than 1.

comment:70 Changed 12 years ago by mkommend

r8477:

  • Added autoregressive target variable Symbol
  • Merged trunk changes into the branch.

comment:71 Changed 12 years ago by mkommend

r8483: Corrected evaluators and time series models.

comment:72 Changed 12 years ago by mkommend

r8487: Added ErrorCharacteristics View for time series models.

comment:73 Changed 12 years ago by mkommend

r8490: Updated TimeSeriesPrognosisSolutionView.

comment:74 Changed 11 years ago by mkommend

r8742: Merged trunk changes and fixed compilation errors due to the merge.

comment:75 Changed 11 years ago by mkommend

r8743: Readded ErrorCharacteristicsView for time series prognosis solutions.

comment:76 Changed 11 years ago by mkommend

r8750: Extracted prognosis results into separate class and added a view for them.

comment:77 Changed 11 years ago by mkommend

r8762: Fixed serialization and corrected AR(n) modeling.

comment:78 Changed 11 years ago by mkommend

r8764: Removed wrong MA models from timeseries prognosis branch.

comment:79 Changed 11 years ago by mkommend

r8765: Fixed minor bugs in time series prognosis views.

comment:80 Changed 11 years ago by mkommend

r8766: Reintegrated trunk updates in timeseries branch.

comment:81 Changed 11 years ago by mkommend

r8767: Merged remaining changes from the trunk.

comment:82 Changed 11 years ago by mkommend

r8768: Corrected plugin dependencies.

comment:83 Changed 11 years ago by mkommend

r8770: Corrected plugin.cs.frame for symbolic time series prognosis views.

comment:84 Changed 11 years ago by mkommend

r8789: Merged trunk changes into timeseries branch.

comment:85 Changed 11 years ago by mkommend

  • Owner changed from mkommend to gkronber
  • Status changed from accepted to reviewing

r8798: Reintegrated time series modeling branch into trunk.

comment:86 Changed 11 years ago by mkommend

  • Owner changed from gkronber to mkommend
  • Status changed from reviewing to assigned

comment:87 Changed 11 years ago by mkommend

r8799: Corrected time series projects (copy local, output path).

comment:88 Changed 11 years ago by mkommend

r8801: Changed project reference to an assembly reference in Problems.DataAnalysis.Symbolic.TimeSeriesPrognosis and updated build dependencies in the solution file.

This change was necessary to allow compilation on the build server.

comment:89 Changed 11 years ago by mkommend

  • Owner changed from mkommend to gkronber
  • Status changed from assigned to reviewing

Only r8798, r8799 and r8801 must be reviewed, because r8798 is the trunk integration and contains all previous changes.

Last edited 11 years ago by gkronber (previous) (diff)

comment:90 Changed 11 years ago by abeham

r8851: Changed assembly reference to a project reference

It always gave me build errors when starting from a fresh checkout. I don't know why it didn't fail on the build server, but it should have.

comment:91 Changed 11 years ago by gkronber

r8852: reverse merge of r8851. Project reference doesn't work on the builder because of a strange path-length bug. see r8801

comment:92 Changed 11 years ago by abeham

Ok, the problem that I was seeing seems to be related to this bug in msbuild. The article says it should be fixed in .Net 4.5.

comment:93 Changed 11 years ago by mkommend

r8940: Added EstimationLimits to ISymbolicTimeSeriesPrognosisModel.

comment:94 Changed 11 years ago by gkronber

Again had to correct revision numbers in previous comments. Please be careful.

comment:95 follow-ups: Changed 11 years ago by gkronber

review comments:

  • fields in TimeSeriesPrognosisResults are not cloned.
  • Update of EvaluatedSolutions is not thread safe in the interpreters.
  • Cannot activate the symbol LaggedVariable in the GUI.
  • the default grammar does only allow real-valued symbols in the root (not time series symbols)
  • node impact calculation should be based on the currently selected evaluator. In particular the currently selected training horizon value should be used. Right now impacts are calculated in the same way as for regression models.
  • It would be nice to also provide a line-chart showing the n-step forecast.
Last edited 11 years ago by gkronber (previous) (diff)

comment:96 Changed 11 years ago by gkronber

r9004:

  • locking for update of EvaluatedSolutions in interpreter
  • cloning of fields in TimeSeriesPrognosisResults
Last edited 11 years ago by gkronber (previous) (diff)

comment:97 Changed 11 years ago by gkronber

I think the training partition for time-series instances should automatically start at 1 or 10 otherwise this has to be changed manually each time (otherwise an exception occurs)

comment:98 Changed 11 years ago by gkronber

r9452: created an override for the Load() method in SymbolicTimeSeriesPrognosisSingleObjectiveProblem to set the first index of the start partition to 10 (prevents ArgumentException)

comment:99 Changed 11 years ago by gkronber

  • Owner changed from gkronber to mkommend
  • Status changed from reviewing to assigned
  • Version changed from branch to 3.3.7

comment:100 Changed 11 years ago by gkronber

Please fix the grammar/symbol problems mentioned in comment:95 for the 3.3.8 release. The other issues should be tracked in new tickets.

comment:101 in reply to: ↑ 95 Changed 11 years ago by mkommend

r9459: Updated TypeCoherentGrammar to take time-series, power, and condition symbols into account.

comment:102 in reply to: ↑ 95 Changed 11 years ago by mkommend

  • Owner changed from mkommend to gkronber
  • Status changed from assigned to reviewing

review comments:

  • Cannot activate the symbol LaggedVariable in the GUI.

All variable symbols can only be activated if at least one input variable is allowed.

  • the default grammar does only allow real-valued symbols in the root (not time series symbols)

This was corrected with r9459. However, I have not updated the special functions.

comment:103 Changed 11 years ago by mkommend

r9466: Fixed SGP symbolic classification tests, whose results got updated, due to a change in the grammar in r9459.

comment:104 Changed 11 years ago by gkronber

  • Status changed from reviewing to readytorelease

Reviewed r9459 and r9466. Thanks!

comment:105 Changed 11 years ago by swagner

  • Resolution set to done
  • Status changed from readytorelease to closed
  • Version changed from 3.3.7 to 3.3.8
Note: See TracTickets for help on using tickets.